Publique en esta revista
Información de la revista
Vol. 10. Núm. 6.
Páginas 364-372 (Noviembre - Diciembre 2014)
Compartir
Compartir
Descargar PDF
Más opciones de artículo
Visitas
3054
Vol. 10. Núm. 6.
Páginas 364-372 (Noviembre - Diciembre 2014)
Original article
DOI: 10.1016/j.reuma.2014.01.010
Acceso a texto completo
Barriers to implementing the “2008 Mexican Clinical Practice Guideline recommendations for the management of hip and knee osteoarthritis” in primary healthcare practice
Barreras para la implementación de la «Guía Mexicana de Práctica Clínica para el manejo de osteoartritis de cadera y rodilla en el primer nivel de atención 2008» en la práctica de atención primaria
Visitas
3054
Adalberto Loyola-Sancheza,
Autor para correspondencia
loyolaa@mcmaster.ca

Corresponding author.
, Julie Richardsona, Ingris Pelaez-Ballestasb, José Guadalupe Sánchezc, Martha Alicia Gonzálezc, Juan Sánchez-Cruzd, María Valeria Jiménez-Baeze, Nancy Nolasco-Alonsof, Idolina Alvaradog, Jacqueline Rodríguez-Amadoh, José Alvarez-Nemegyeii, Mike G. Wilsonj
a School of Rehabilitation Science, McMaster University, Hamilton, Canada
b Departamento de Reumatología, Hospital General de México «Eduardo Liceaga», México D.F., Mexico
c Centro Universitario de Salud, Universidad Autónoma de Nuevo León, Monterrey, Mexico
d Instituto Mexicano del Seguro Social, Delegación Estatal Yucatán, Mérida, Mexico
e Unidad de Medicina Familiar No. 14-Quintana Roo, Instituto Mexicano del Seguro Social, Cancún, Mexico
f Unidad de Medicina Familiar No. 60, Instituto Mexicano del Seguro Social, Estado de México, Mexico
g Centro de Salud “Independencia”, Servicios de Salud Morelos, Jiutepec, Mexico
h Hospital Universitario de Nuevo Leon “Jose Eleuterio Gonzalez”, Monterrey, Mexico
i Escuela de Medicina. Universidad Anáhuac-Mayab, Mérida, Mexico
j McMaster Health Forum, Department of Clinical Epidemiology and Biostatistics, McMaster University Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, Canada
Este artículo ha recibido
3054
Visitas
Información del artículo
Resumen
Texto completo
Bibliografía
Descargar PDF
Estadísticas
Figuras (2)
Tablas (5)
Table 1. Examples of reviewers’ analytic interpretations.
Table 2. 2008-Mexican-OA-CPG's Global Quality Evaluation.
Table 3. General and context-related barriers for implementation and possible solutions: Medical History (CH) and Non-pharmacologic treatment (NPT).
Table 4. General and context-related barriers for implementation and possible solutions: Pharmacologic treatment (PT), Technical Aids and Orthotics (TAO), and Referral (R).
Table 5. Recommendations considered as ready to be implemented in clinical practice.
Mostrar másMostrar menos
Material adicional (1)
Abstract
Objective

To evaluate the implementability of the “2008 Mexican Clinical Practice Guideline for the management of hip and knee osteoarthritis at the primary level of care” within primary healthcare of three Mexican regions using the Guideline Implementability Appraisal methodology version 2 (GLIA.v2).

Methods

Six family physicians, representing the South, North, and Central Mexico, and one Mexican physiatrist evaluated the 45 recommendations stated by the Mexican guideline. The GLIA.v2 methodology includes the execution of qualitative and semi-quantitative techniques.

Results

Reviewers’ agreement was between moderate to near complete in most cases. Sixty-nine percent of the recommendations were considered difficult to implement within clinical practice. Eight recommendations did not have an appropriate format. Only 6 recommendations were judged as able to be consistently applied to clinical practice. Barriers related to the context of one or more institutions/regions were identified in 25 recommendations. These barriers are related to health providers/patients’ beliefs, processes of care within each institution, and availability of some treatments recommended by the guideline.

Conclusions

The guideline presented problems of conciseness and clarity that negatively affect its application within the Mexican primary healthcare context. We identified individual, organizational and system characteristics, which are common to the 3 institutions/regions studied and constitute barriers for implementing the guideline to clinical practice. It is recommended that the 2008-Mexican-CPG-OA be thoroughly revised and restructured to improve the clarity of the actions implied by each recommendation. We propose some strategies to accomplish this and to overcome some of the identified regional/institutional barriers.

Keywords:
Osteoarthritis
Hip
Osteoarthritis
Knee
Practice guidelines as Topic
Mexico
Abbreviations:
2008-Mexican-CPG-OA
TAO
COX-2
CPG
dec
exe
fle
GLIA
CH
IMSS
mea
nov
NSAIDs
OA
pro
R
SS
PT
NPT
val
Resumen
Objetivo

Evaluar las barreras de implementación de la guía de práctica clínica para el manejo de osteoartritis de cadera y rodilla en el primer nivel de atención 2008 dentro de la práctica clínica de 3 regiones mexicanas, usando la metodología Guideline Implementability Appraisal version 2 (GLIA v2).

Métodos

Seis médicos familiares, representantes del sur, norte y centro de México, y un médico rehabilitador mexicano evaluaron las 45 recomendaciones propuestas en la guía de práctica clínica. La metodología GLIA v2 incluye la ejecución de técnicas cualitativas y semicuantitativas.

Resultados

En su mayoría, el acuerdo entre revisores fue de moderado a casi completo. El 69% de las recomendaciones fueron consideradas como difíciles de implementar en la práctica clínica. Ocho recomendaciones no tienen un formato apropiado. Únicamente 6 recomendaciones pueden ser aplicadas consistentemente en la práctica clínica. En 25 recomendaciones, se detectaron barreras de implementación relacionadas al contexto de una o más de las instituciones/regiones exploradas. Estas barreras se relacionan con las creencias de proveedores de salud y pacientes, procesos de atención en cada institución y disponibilidad de algunos de los tratamientos recomendados en la guía.

Conclusiones

La guía contiene recomendaciones poco claras y concisas, lo que afecta negativamente a su aplicación dentro del primer nivel de atención mexicano. Identificamos características individuales, organizacionales y sistemáticas, comunes a las 3 instituciones/organizaciones estudiadas, que significan barreras para implementar la guía en México. Se recomienda que esta guía sea revisada y reestructurada con el fin de mejorar la claridad de sus recomendaciones. Proponemos algunas estrategias para hacer esto y atacar algunas de las barreras identificadas relacionadas dentro de las regiones exploradas.

Palabras clave:
Osteoartritis
Cadera
Osteoartritis
Rodilla
Guías de práctica clínica como tópico
México
Texto completo
Introduction

Osteoarthritis (OA) is a chronic musculoskeletal disease of the joints that has a negative impact on the healthy aging of the population.1 This chronic condition produces disability resulting in significant costs to the society.2 It is estimated that in Mexico 10.5% of the people with musculoskeletal pain have OA,3 the knee being the most commonly affected joint.4–6 The high prevalence of OA within the Mexican population produces a high demand for healthcare, representing one of the 10 most common problems seen at the primary level of care in the Mexican Institute of Social Security (IMSS), one of the main public health institutions of the country.7

In Mexico, people with OA are usually managed by general and family physicians within the primary healthcare system,4 representing one of the principal sources of cost for this level of care.8 Consequently, there is an interest in containing the high social and economic costs produced by this chronic disease through development and implementation of clinical practice guidelines (CPG) that standardize management across the country.9 Clinical practice guidelines are systematically developed statements or recommendations that assist in the decision-making process within healthcare.10 In 2008, the Mexican Secretary of Health coordinated the development of a CPG for the management of knee and hip OA at the primary level of care with the main purpose of providing evidence-based recommendations to decrease the disabling effects of hip and knee OA in the Mexican population.11 The developers of this guideline used existing CPGs from other countries to structure their recommendations for practice.9

The CPGs used to create the “2008 Mexican Clinical Practice Guideline for the management of knee and hip OA at the primary level of care (2008-Mexican-CPG-OA)” have not been fully implemented within their own contexts.12–15 This situation raises questions about the direct transferability of the recommendations stated by the 2008-Mexican-CPG-OA to the Mexican context. According to the Knowledge-To-Action (KTA) framework, in order to successfully apply a knowledge tool such as a CPG, it is important to identify potential barriers to its implementation, considering the local context in which it will be utilized.16

The implementability of a CPG refers to a set of its recommendations’ characteristics that permit their successful conversion into actions.17 Only clear, concise, and actionable recommendations can be successfully implemented in clinical practice.18,19 In consequence, it is possible to assess the barriers to implementation of a CPG through analyzing the characteristics of its recommendations.

The Mexican Public Health system is formed by different institutions, such as the IMSS and the Secretary of Health (SS), each with its own government structures and procedural mechanisms. Even within each institution there are region-related structural and systemic differences. This situation underlies the complexity of the public Mexican Healthcare system, implying the presence of different healthcare contexts that could affect the implementability of the 2008-Mexican-CPG-OA.

As a result, the idea behind this study was to use the concept of “CPG implementability” to evaluate the recommendations proposed in the 2008-Mexican-CPG-OA. The main objectives were to evaluate the implementability of the 2008-Mexican-CPG-OA within different Mexican Healthcare institutions at the primary level of care in three Mexican regions (Northern, Central, and Southern) and to put forward some strategies to improve its successful implementation within clinical practice in Mexico. To accomplish this, we used the GLIA v2 instrument to: (a) identify implementation barriers for each recommendation of the guideline, (b) disclose differences on implementability issues among each institution and region, and (c) propose strategies to address the identified barriers.

MethodsDesign overview

This was a consensus-based exercise that used qualitative and semi-quantitative techniques, following the methodology proposed by the Yale Center for Medical Informatics known as the Guideline Implementability Appraisal version 2 (GLIA v2).17 Six family physicians and one physiatrist collaborated to evaluate the barriers for the implementation of the recommendations stated by the 2008-Mexican-CPG-OA within their clinical practices.

Reviewers

The family physicians formed three teams representing different geographical regions and institutions: (1) the Northern team formed by two physicians from the SS-Monterrey, Nuevo León; (2) the Central team formed by one physician from the SS-Jiutepec, Morelos, and one physician from the IMSS-Tlalnepantla, Estado de México; and (3) the Southern team formed by one physician from the IMSS-Mérida, Yucatán, and one physician from the IMSS-Cancún, Quintana Roo. These physicians had at least one-year experience treating patients with knee and/or hip OA, had sufficient English language skills to follow the GLIA methods, and were prepared to commit enough time to complete the project. The physiatrist (AL) is from Mexico City and was in charge of the other reviewers’ training on the GLIA v2 application. In addition, he facilitated and coordinated communications within and across the teams. All family physicians completed a questionnaire inquiring about the following: a) their level of experience with health research, b) their previous knowledge about the existence of the 2008-Mexican-CPG-OA, c) their previous training on this CPG application, d) their previous training on OA management, and e) the percentage of weekly consultations dedicated to manage people with OA.

2008-Mexican-CPG-OA

The Mexican Secretary of Health coordinated the development of the 2008-Mexican-CPG-OA, collaborating with other Mexican public institutions. The main objective was to standardize the management of hip and knee OA within primary care, providing the best evidence available to manage the progression and disabling effects of these chronic conditions in Mexico.11 The recommendations stated in this CPG came from statements found in 5 international CPGs (1 Latin American, 1 North American, 2 Western European and 1 multinational)20–24 and 1 program implementation study.25

The Mexican CPG contains 45 recommendations structured in 5 main sections: “Medical History”, “Non-pharmacologic treatment”, “Pharmacologic treatment”, “Technical Aids and Orthotics”, and “Referral to a secondary level of care”. The “Non-pharmacologic treatment” section is further divided into: physical agents, weight reduction, therapeutic exercise, electrotherapy, and dietary supplements. The “Pharmacologic treatment” section is further divided into oral analgesics, and topical analgesics.

GLIA v2

The GLIA v2 instrument was developed to systematically identify barriers to implement CPGs’ recommendations.17 This instrument was created from a broad review of the factors that impact the success of guidelines’ use within the literature,17 and has been utilized to improve guideline recommendations.26 The GLIA is formed by 30 questions, 9 to evaluate global quality and 21 to assess the barriers for implementation of each individual recommendation. Details of the content and execution of this instrument are provided elsewhere.27 Briefly, each recommendation is assessed for its executability, decidability, validity, flexibility, effect on the process of care, measurability, novelty, and computability. We did not use the computability dimension, because we were not interested in the implementation of the Mexican CPG in electronic format.

Procedure for the guideline's evaluation

We coded the 45 recommendations, giving each a unique identification number (see Appendix A). Initially, each team member worked individually on the guideline evaluation and then discussed his/her results with the other member of his/her team. The physiatrist moderated this discussion in person. All communications were conducted either through personal meetings, televideo conferencing or electronic mail during 4.5 months (see Fig. 1). Disagreements were solved through two rounds of discussion and the application of the “70% agreement” rule, meaning that each member expressed at least 70% satisfaction with what was agreed.28 All reviewers gave permission to audio-record the meetings. The global quality of the guideline was determined through consensus from all members of all the teams because the guideline's overall quality depends on its consideration to the Mexican context as a whole.

Fig. 1.

Procedure for the guideline's evaluation.

(0,26MB).
Analysis

Agreement between each team of reviewers was evaluated by calculating nominal data kappa coefficients29 using an on-line tool.30 We used these coefficients to judge agreements as follows: poor=<0.2, fair=0.21–0.4, moderate=0.41–0.6, strong=0.61–0.8, and near complete=>0.8.29 We assessed each team's responses to GLIA questions 10 to 26 in all 45 recommendations.

The identified barriers to implementation were classified as either general or context-related. General barriers were defined as those related to the intrinsic structure of a recommendation; these are barriers for the executability, decidability, validity, flexibility and measurability of a recommendation.27 Context-specific barriers were defined as those related to the teams’ institution or region and are barriers related to novelty of the recommendation or to the process of care of each specific site.27 A synthesis of each recommendation's barriers for implementation, along with strategies for addressing them, was then created. All team members reviewed this synthesis and confirmed its validity.

Following GLIA recommendations, we considered all recommendations with executability and/or decidability barriers as “not possible to implement” in a consistent way within clinical practice. In addition, we considered that all recommendations without a clear action implied within their statements should be eliminated from the guideline or transformed into an implementable recommendation. Refer to Table 1 for some examples of analytic interpretations performed by reviewers.

Table 1.

Examples of reviewers’ analytic interpretations.

Statement without a recommendation format
PT8: There is no evidence to support the use of tricyclic anti-depressants in osteoarthritis pain
Interpretation  Reviewers agreed that there is no action implicated in this statement; hence, this is not a recommendation and should be eliminated from the guideline. 
Recommendation with general barriers
PT10: Paracetamol should be considered for the management of osteoarthritis, after the use or in continuity with NSAIDs, COX-2 or opioids
Interpretation  Reviewers agreed that this recommendation has executability problems because the circumstances in which paracetamol prescription goes “after” or goes “along with” NSAIDs, COX-2 or opioids are not clear. There are no details on the paracetamol dosage. The consistent implementation of this recommendation is not possible. This recommendation has validity issues because justification for the recommendation is not explicitly stated in the guideline. This recommendation contradicts the information given in the quick guide in which paracetamol appears as the first choice of pharmacological treatment for knee and hip OA. 
Recommendation with context-specific barriers
NPT14: Electro-acupuncture should not be used in the management of people with osteoarthritis
Interpretation  SS-Monterrey reviewers agreed that this recommendation has novelty problems because it goes against the beliefs of some people living in Monterrey. The use of electro therapy and acupuncture is fairly common; therefore, it is important to design patient education strategies directed to avoid the use of these therapeutic modalities, providing a strong and clear justification. 

GLIA, Guideline Implementability Assessment Instrument; PT8, recommendation number 8 from the “Pharmacologic treatment” section; NSAIDs, Non-steroid anti-inflammatory drugs; COX-2, cyclooxygenase 2 inhibitors; PT10, recommendation number 10 from the “Pharmacologic treatment” section; NPT14, recommendation number 14 from “Non-pharmacologic treatment” section; SS, Monterrey-Mexican Health Secretary of Monterrey.

Results

The reviewers had been practicing family medicine for an average of 17 years (min – 7, max – 27) and dedicate 5–30% of their weekly clinical time to the management of OA. All reviewers were familiar with evidence-based practice concepts, and 4 reviewers were undertaking postgraduate studies in health research methodology. Two reviewers were aware of the existence of the 2008-Mexican-OA-CPG before starting this project; none of them had received training for implementing this guideline. Only one reviewer, from the Northern team, had taken formal training on OA management.

There was more moderate to near-complete agreement (n=81) than poor to fair agreement (n=41) in the assessment of each recommendation within all the teams (see Fig. 2). This trend did not change when considering the agreement of each team separately. Some calculated kappa coefficients were negative.

Fig. 2.

Frequencies of poor/fair and moderate/near-complete agreement between reviewers.

(0,15MB).

All teams agreed that the guideline clearly defined the target population and audience and has a format that allows for an easy identification of the recommendations. However, we detected important problems with the settings’ description, the guideline developers’ representativeness, and the recommendations’ sequence, internal consistency and conciseness (see Table 2).

Table 2.

2008-Mexican-OA-CPG's Global Quality Evaluation.

GCQa  Answer  Comment 
The target population is clearly identified in the document. 
The target audience is clearly identified in the document. 
The settings in which several recommendations should be applied are not described. Some recommendations are not even applicable to the Mexican primary healthcare context. It is necessary to provide a clear description of each recommendation's setting in order to assure the consistency of the guideline's application. 
Health providers such as, rheumatologists, orthopedic surgeons, nurses, physiotherapists, occupational therapists, and users were not represented in the group of guideline developers. 
The guideline includes a “quick guide”. However it is not very useful, because it includes information that contradicts the full version, and includes information not provided in the complete guideline. 
There is no sequential logic in the form the recommendations are presented. Some recommendations are repetitive, affecting the understandability of the guideline. It is important to build a sequence in the recommendations; the algorithm provided in the quick version can help with this structuring. 
There are some contradictions between the evidence presented and the recommendations stated. There are evidence statements without recommendations and recommendations without evidence statements. 
The guideline format allows for an easy identification of the recommendations. 
Several recommendations are ambiguous regarding the action contained. There are many recommendations with repetitive information. 

2008-Mexican-OA-CPG, 2008 Mexican Clinical Practice Guideline for the management of knee and hip OA at the primary level of care; GCQ, global considerations questions; Y, yes; N, no.

a

1. Does the guideline clearly define the target population? 2. Does the guideline clearly define its intended audience? 3. Are the settings in which the guideline is to be used clearly described? 4. Do the organizations and authors who developed the guideline have credibility with the intended audience? 5. Does the guideline suggest strategies for implementation or tools for application? 6. Is it clear in what sequence the recommendations should be applied? 7. Is the guideline internally consistent? 8. Are all recommendations easily identifiable? 9. Are all recommendations concise?

The general and context-related barriers for implementation identified for each recommendation, along with their proposed solutions, are described in Tables 3 and 4. Thirty-one recommendations presented executability and/or decidability problems that make them difficult to be applied consistently to clinical practice. Eight recommendations were just statements which contained no actions, so they do not have an appropriate recommendation format. Only six recommendations were judged as appropriate to be applied consistently to clinical practice in their current format (see Table 5).

Table 3.

General and context-related barriers for implementation and possible solutions: Medical History (CH) and Non-pharmacologic treatment (NPT).

Guideline section  Barriers
  Ga  Proposed solutionsb 
Medical History  exe fle val  None  Define in: 1, how to assess “activity limitations” and conduct “physical assessment”; 3, meaning of “high doses” and “chronic use”. Change “warfarin” for “anticoagulants” in 2. Clarify meaning of “Age60 years” in 3. Provide justification for 1, 2 and 3
Non-pharmacologic treatment: general  exe fle val dec  pro nov mea  2 and 3 are not recommendations. Create: one recommendation from 3 defining “good communication” and expected outcomes; three recommendations from 4; two recommendations from 5 detailing “weight reduction” “joint protection” “education” and “advice” and conditioning to overweight or obese people. Define in:1, how to do “joint decision making” and what to do for people with no decision capacity; 6, content of “exercises”, “self-management”, “physical” and “occupational therapy”, “functional limitations”, what to do with people without functional limitations, and who is responsible for prescription/execution. Provide justification for 1, 4, 5 and 6.In the 3 regions/institutions: Design training strategies for exercise prescription, self-management and OA rehabilitation. Allow more time per consult. Build or improve existing primary care rehabilitation units. 
Non-pharmacologic treatment: physical agents  exe fle val  pro novmea  For 7: Define details on “cryotherapy” and “superficial heat” prescription; who should receive these interventions and for what purpose; and who is responsible for prescribing and applying them. Provide justification for this recommendation.In the 3 regions/institutions: Design training strategies on thermotherapy prescription. 
Non-pharmacologic treatment: weight reduction  exe fle val dec  None  Merge8 and 9. Define how much weight reduction is recommended; monitoring strategies; meaning of “low levels”; and person's characteristics that allow individualization. Provide justification. Include persons with obesity. 
Non-pharmacologic treatment: therapeutic exercise  exe fle val  pro nov  10 is not a recommendation. Merge first part of 12 with 11 and create a single recommendation for “pool exercises”. Define prescription details and who is responsible for prescribing this. Provide justification.In the 3 regions/institutions: Build primary care rehabilitation units including therapeutic pools. Design training strategies for exercise prescription. 
Non-pharmacologic treatment: electrotherapy  exe fle val  pro nov mea  Define in 13, TENS prescription details and who is responsible for it. Provide justification for using indirect evidence to state 14 (evidence for acupuncture was extrapolated to electro-acupuncture).In the 3 regions/institutions: Design training strategies on TENS prescription and assure devices’ availability. In SS-Monterrey: Design patient-education strategies to avoid use of electro-acupuncture. 
Non-pharmacologic treatment: dietary supplements  exe val  pro  For 15Define “glucosamine” and “condroitin sulfate” dosages. Provide justification for recommending an intervention that is “no better than placebo” as described by the “evidence boxes” of the guideline.In the 3 regions/institutions: Assure the availability of glucosamine and condroitin sulfate. 

CH, Clinical History; NPT, Non-pharmacologic treatment; G, general barriers; C, context-related barriers; exe: executability (recommendation says exactly what to do); fle: flexibility (degree to which a recommendation permits interpretation allowing for execution alternatives); val: validity (degree to which a recommendation reflects the intent of developers and strength of evidence); dec: decidability (recommendation says precisely under what conditions to do something); pro: effects on process of care (degree to which a recommendation impacts upon the usual workflow in a typical care setting); nov: novelty (degree to which a recommendation proposes unconventional behaviors for clinicians or patients); mea: measurability (degree to which markers of a recommendation's effects are available); TENS: Transcutaneous Electrical Neuro Estimulation.

a

The strength of all recommendations is not explicitly stated.

b

Recommended actions within solutions are in italics. Bold numbers identify each recommendation within each guideline's main section (refer to Appendix 1, “The coded 2008-Mexican-CPG-OA” to read each recommendation's content).

Table 4.

General and context-related barriers for implementation and possible solutions: Pharmacologic treatment (PT), Technical Aids and Orthotics (TAO), and Referral (R).

Guideline section  Barriers
  Ga  Proposed solutionsb 
Pharmacologic treatment: oral analgesics  exe fle val dec  pro nov  8 is not a recommendation. Describe how the strength of 1 was judged. Define in: 2, action proposed, how to monitor “risk factors, and meaning of “adequate monitoring”; 3, meaning of “risks for NSAIDs”, actions proposed for “not at risk” people and persons’ characteristics that allow for individualization; 4, when to use “NSAIDS” and when “COX-2” inhibitors considering those patients in whom both paracetamol and topical NSAIDs are not effective; 5, meaning of “minor effective doses” and “shorter periods of time” and patients’ characteristics that allow for individualization; 6, meaning of “risk for NSAIDs” and what to do with people at risk for NSAIDs but no gastrointestinal bleeding; 7, meaning of “cardiovascular diseases” and “being very careful”; 9, opioids and narcotics dosage and patients’ characteristics for individualization; 10, paracetamol dosage and in what circumstances this is used “after” or “along with”; 11, meaning of “risk” and “benefits” and how to assess these; 12, how to modify paracetamol doses and monitor “risk for toxicity”; 13, dosage of all interventions and patients’ characteristics that allow for individualization. Explain in: 3, why a previous imperative recommendation (1) is converted into a conditional recommendation; 6, if the intention is to prescribe COX-2 inhibitors during an active gastrointestinal bleeding episode; 7, why the evidence provided on “diclofenac” extends to all NSAIDs; 10, the contradiction with the “quick guide” in which paracetamol is the first line of pharmacological treatment. Provide justification for 4, 5 and 13. Divide:7 in two: 7a for “cardiovascular diseases” and 7b for “minimal effective doses” and “gastric protector”; 12 in two: 12a for “paracetamol dosage” and 12b for “toxicity modification”. Merge:7b with 1, 3 and 5; and 11 with 9.In the 3 regions/institutions: Assure availability of COX-2 inhibitors, weak opioids and narcotics. Design prescription-training strategies. In SS-Monterrey: Design dissemination strategies to prescribe COX-2 during active gastrointestinal bleeding. 
Pharmacologic treatment: topical analgesics  exe fleval dec  pro  17 is not a recommendation. Define in: 14, capsaicin dosage and indications; 15 and 16, which topical NSAIDs should be considered, what doses and how can they be combined with “basal treatment” and paracetamol; 18-how capsaicin should be combined with “basal treatment”. Merge: 15 with 16 and 14 with 18. Provide justification for 14, 15, 16, 18 and 19. Explain why 15, 16 and 18 are conditioned only to knee OA including alternatives for hip OA. In IMSS-Merida, Cancun, and Edo. Mex: Assure availability of topical capsaicin and NSAIDs. 
Technical Aids and Orthotics  exe fle val  pro  4 and 5 are not recommendations. Create two recommendations on the use of “splinting” and “shoe insoles” defining prescription details. Define in: 1, which “orthoses”, patients’ characteristics that allow individualization and who is in charge of prescribing them; 2, further instructions for “cane and walker” prescription allowing for individualization; 3, meaning of “appropriate footwear”, and who is responsible for prescription.In the 3 regions/institutions: Design training strategies for orthoses, cane/walker, and footwear prescription. Assure availability of orthoses, canes and walkers. Build or improve existing primary care rehabilitation units. 
Reference to 2nd level of care  fle val dec  pro nov  3 is not a recommendation. Explain why only patients with functioning states II and IV should be referred to a 2nd level of care. Define in 2 meaning of “adequate pain relief” and “functional improvement” allowing for individualization. Provide justification for 2adding the “referral” action.In the 3 regions/institutions: Design training strategies for “functional capacity” evaluation, and patient education strategies to change negative beliefs around joint replacement. Improve efficiency of referral processes. 

PT, Pharmacologic treatment; TAO, Technical Aids and Orthotics; R, Referral to second level of care. G, general barriers; C, context-related barriers. exe: executability (recommendation says exactly what to do); fle: flexibility (degree to which a recommendation permits interpretation allowing for execution alternatives); val: validity (degree to which a recommendation reflects the intent of developers and strength of evidence); dec: decidability (recommendation says precisely under what conditions to do something); pro: effects on process of care (degree to which a recommendation impacts upon the usual workflow in a typical care setting); nov: novelty (degree to which a recommendation proposes unconventional behaviors for clinicians or patients); mea: measurability (degree to which markers of a recommendation's effects are available); NSAIDs: Non-Steroid Anti-Inflammatory Drugs; COX-2: Cyclooxygenase 2; SS-Mexican Health Secretary; IMSS, Mexican Institute of Social Security.

a

The strength of all recommendations is not explicitly stated.

b

Recommended actions within solutions are in italics. Bold numbers identify each recommendation within each guideline's main section (refer to Appendix 1, “The coded 2008-Mexican-CPG-OA” to read each recommendation's content).

Table 5.

Recommendations considered as ready to be implemented in clinical practice.

Rec ID  Recommendation 
CH2  “The gastrointestinal risk evaluation in people with OA should include: (a) Gastrointestinal bleeding history, (b) Peptic ulcer history or steroid-induced gastrointestinal symptoms, (c) Current use of corticosteroids and warfarin.” 
NPT14  “Electro acupuncture should not be used in the management of people with osteoarthritis.” 
PT1  “When osteoarthritis is treated with oral NSAIDs or COX-2 inhibitors, a proton-pump inhibitor should be prescribed, selecting the least costly.” 
PT19  “Topic rubefacients (trolamine salicylate and cooper salicylate) are not recommended for the management of osteoarthritis.” 
TAO2  “Walking aids reduce pain in patients with hip and knee osteoarthritis. Patients should be instructed on the optimal use of a cane with the arm opposite to the affected joint. Walkers with wheels are recommended in the cases of bilateral affection.” 
R1  “Patients with hip and knee osteoarthritis with functional class III and IV (ARA) should be referred to a rehabilitation unit to receive an evaluation and program prescription from a physician specialized in Physical Medicine and Rehabilitation, who will use different therapeutic modalities such as electrotherapy or therapeutic exercises to decrease their pain and improve their functional capacity. A successful rehabilitation program could avoid the need of walking aids such as canes and walkers.” 

Rec ID, recommendation ID (see Appendix A); CH, Clinical History; NPT, Non-pharmacologic treatment; PT, Pharmacologic treatment; TAO, Technical Aids and Orthotics; R, reference to the 2nd level of care. NSAIDs, non-steroid anti-inflammatory drugs; COX-2, cyclooxygenase 2 inhibitors; ARA, American Rheumatism Association (Now known as American College of Rheumatology (ACR)).

The three teams agreed that the guideline does not provide a clear description of how the strength of the recommendations was determined. Although the guideline mentions the strength for each recommendation, the criteria used to adjudicate this strength are not explained in the document. The nomenclature to describe recommendations’ strength is inconsistent; sometimes it is through letters and sometimes through numbers. In addition, information about how the guideline developers chose the recommendations from the different CPGs used to create this guideline is also not described in the document.

Twenty-five recommendations were identified with barriers related to the context of one or more of the institutions-regions represented in this project. The Northern team identified barriers mainly related to health providers’ beliefs and patients’ expectations. The Central team identified barriers related to the process of care and treatment availability. The Southern team identified barriers related to the process of care, and patients’ expectations. We also identified common barriers related to the context of the three institutions–regions represented in the study (see Tables 3 and 4).

Discussion

This study uncovered some aspects of the 2008-Mexican-CPG-OA that can impede its successful implementation within the IMSS-Estado de México, Quintana-Roo and Yucatán, and in the SS-Morelos and Nuevo León. As a whole, the guideline presented problems of conciseness and clarity that negatively affect its credibility and application within the Mexican primary healthcare context. Only 6 of the 45 recommendations (14%) were considered to be implementable in a consistent way. We also detected differences and similarities in the identified context-related barriers for implementation among the three institutions and regions represented in this project. Finally, we were able to propose some strategies to increase the guideline's successful implementation within Mexican clinical practice.

Clinical practice guidelines synthesize and inform the best available knowledge to support decision-making processes in clinical healthcare. The developers of the 2008-Mexican-CPG-OA created this guideline not only to aid in the decision-making process of primary caregivers but also to “standardize national actions” regarding the management of knee and hip OA in Mexico.11 The standardization of clinical practice at a national level could be an ambitious goal for a CPG. The guideline presents some issues regarding its structure, recommendations’ clarity and developers’ credibility, all of which may affect its utility for supporting decisions, making the standardization of clinical practice within the primary care of Mexico difficult.

The guideline's structure does not allow for a clear understanding of the settings and the sequence in which recommendations should be applied. It has been observed that CPGs without comprehensible structure are not easy to implement during everyday practice.31 Furthermore, most of the recommendations stated in this guideline are neither consistent nor concise, and it is difficult to understand the required actions implicit within them. The literature shows that in practice, vague recommendations are significantly less utilized than recommendations that clearly state what to do.32 On the other hand, reviewers expressed concern about the guideline developers’ credibility, noting that some knowledge users and patients were not involved in the guideline creation. A literature review on OA management conducted in Mexico, concluded that management of people with OA should be multi-disciplinary, including rheumatologists, orthopedic surgeons and nutriotionists,33 none of whom were represented in the creation of the 2008-Mexican-CPG-OA.

Increasing evidence shows that involving patients and knowledge users in the development of a CPG improves its applicability to clinical practice.34,35 Consequently, we think it is necessary to restructure the 2008-Mexican-CPG-OA replacing ambiguous recommendations for behavior-specific statements that clarify the what, who, when, where and how of the intended actions.36,37 This restructuring process should also include representatives from patients, primary care clinicians, rheumatologists and orthopedic surgeons. We also suggest that the quick guide (see Table 2) should be restructured, ensuring that its content matches the contents of the complete guideline.

The vast majority of the guideline's recommendations (86%) presented executability and/or decidability problems, which implies great challenges for its consistent implementation to the Mexican context. Failed executability and decidability are often the result of vagueness in the description of the actions involved in a recommendation.38 This vagueness results in application inconsistencies that go against the standardization purposes of a guideline. Interestingly, the 2008-Mexican-CPG-OA recommendations preserved the original foreign CPGs’ format. This may suggest that problems observed in the implementation of some of these foreign CPGs in countries such as France,13,39 Canada,12 and the UK15 are due to executability and decidability issues.

Only two of the six recommendations judged to be implementable in their current form (see Table 5) address one of the eight actions recently proposed as the minimum standard of care for people with knee and hip OA, “to minimize individual risk potential for NSAIDs harms”.40 The Mexican CPG considers all of these standard actions of care. This suggests that a simple restructuring of the recommendations can potentially result in an improvement in the quality of care for people living with OA in México.

Apparently, the values used to define the strength of the recommendations in the 2008-Mexican-CPG-OA were based on study design. Nevertheless, it has been widely recognized in the literature that the strength of CPG recommendations should not only reflect the quality of supporting evidence, but also the values and beliefs of developers, considering the context.41–43 Moreover, it has been strongly recommended that the processes to determine the strength of a recommendation be described in a transparent way.19 Consequently, we urge the guideline developers to be more explicit about the procedures they followed to state the recommendations’ strength, so the audience can judge their validity.

The context-related barriers identified can be further categorized as individual, organizational or system related.18 The Northern team identified individual barriers, the Central team identified organizational and system-related barriers, and the Southern team identified individual, organizational and system-related barriers. This shows how the context could influence the implementability of a recommendation, supporting the argument that CPGs have to be adapted to local conditions.44 Furthermore, the identification of organizational and system-related barriers suggests that not considering local socioeconomic and political factors may result in the failure to implement the CPG.

We also found similarities regarding context-related barriers identified by the three teams, suggesting the existence of implementability issues in the whole Mexican public primary care health system. These barriers were related to individual factors (lack of family physicians’ skills for prescribing rehabilitation interventions or evaluating functional capacity, and patients’ negative beliefs about arthroplasty); organizational factors (insufficient time and inefficient referral processes); and system-related factors (absence of efficient primary care rehabilitation units, therapeutic pools, and therapeutic agents). These implementation barriers can be addressed through patient education and knowledge dissemination strategies for health providers. Moreover, organizational and system changes require negotiation with local, regional and federal health administration and policy representatives.

There were some issues during the application of the GLIA v2 methodology in this study. All reviewers had problems with the concept of flexibility. Flexibility is defined as “the degree to which a recommendation permits interpretation and allows for alternatives in its execution”.45 This seems to be opposed to the idea of “consistency” evaluated in the executability dimension. After some discussion, team members concluded that “flexibility” favors the implementability of a recommendation allowing for consistent tailoring and does not imply an “anything goes” approach. Moreover, all teams agreed that the recommendation's strength does not belong to “flexibility” but to the validity dimension of the GLIA v2. It was difficult for some reviewers to differentiate between conditional and imperative recommendations and for discerning which statements were not recommendations. These difficulties resulted in negative kappa coefficients during agreement analyses, meaning less agreement than expected by chance.29

In spite of this, all reviewers agreed that the application of the GLIA v2 instrument, although time-consuming, was very helpful to stimulate consideration about guideline implementation barriers, allowing them to reflect on their clinical practices. We think that the GLIA v2 methodology can be further improved by (1) clarifying the dimension of flexibility, (2) providing examples that demonstrate differences among dimensions, (3) providing examples that show the difference between conditional and imperative recommendations, and (4) adding criteria to allow for the identification of statements, which are not recommendations.

One of the limitations of this study is the small number of physicians involved in the guideline evaluation process, which affects the generalizability of the results. However, the 6 family physicians involved had experience managing people with knee or hip OA, representing the two largest public health institutions of Mexico across 3 large geographical regions. Moreover, involving physicians who were currently in research training ensured the quality of the evaluations and analyses. Another limitation is that we did not evaluate the actual success in the implementation of the recommendations at the primary level of care. Consequently, we deduced that the 2008-Mexican-CPG-OA is not consistently implemented in clinical practice due to problems with the executability and decidability of its recommendations. Obviously, this deduction needs to be further evaluated.

In conclusion, the 2008-Mexican-CPG-OA has important barriers to its consistent implementation within the IMSS of Estado de México, Quintana-Roo and Yucatán, and in the SS of Morelos and Nuevo León. We recommend that this guideline undergo thorough revision and restructuring to improve the clarity of the actions implied in each recommendation. We propose some strategies to facilitate this revision, along with some implementation activities to address individual, organizational and system-related barriers. These efforts will potentially increase the success in the 2008-Mexican-CPG-OA's implementation, improving the standard of care for Mexican people living with knee and hip osteoarthritis.

Ethical disclosuresProtection of human and animal subjects

The authors declare that no experiments were performed on humans or animals for this investigation.

Confidentiality of data

The authors declare that no patient data appear in this article.

Right to privacy and informed consent

The authors declare that no patient data appear in this article.

Conflicts of interest

All authors declare no conflicts of interest.

Acknowledgements

We thank Dr. Thelma Martinez Villareal, Dr. Mario Garza and Dr. Jorge Esquivel Valerio from the research department of the School of Medicine at the Universidad Autónoma de Nuevo León for their invaluable input and support during the conduction of this study. Adalberto Loyola-Sanchez is a recipient of a CONACYT (Consejo Nacional de Ciencia y Tecnología) scholarship for foreign studies and a CIHR (Canadian Institute for Health Research) Vanier scholarship. Travel expenses were partially covered by a School of Rehabilitation Science graduate bursary.

Appendix A
Supplementary data

The following are the supplementary data to this article:

References
[1]
Y. Zhang, J.M. Jordan.
Epidemiology of osteoarthritis.
Rheum Dis Clin North Am, 34 (2008), pp. 515-529
[2]
S. Gupta, G.A. Hawker, A. Laporte, R. Croxford, P.C. Coyte.
The economic burden of disabling hip and knee osteoarthritis (OA) from the perspective of individuals living with this condition.
Rheumatology (Oxford), 44 (2005), pp. 1531-1537
[3]
I. Pelaez-Ballestas, L.H. Sanin, J. Moreno-Montoya, J. Alvarez-Nemegyei, R. Burgos-Vargas, M. Garza-Elizondo, et al.
Grupo de Estudio Epidemiologico de Enfermedades Musculo Articulares (GEEMA). Epidemiology of the rheumatic diseases in Mexico. A study of 5 regions based on the COPCORD methodology.
J Rheumatol, 86 (2011), pp. S3-S8
[4]
M.H. Cardiel, J. Rojas-Serrano.
Community based study to estimate prevalence, burden of illness and help seeking behavior in rheumatic diseases in Mexico City. A COPCORD study.
Clin Exp Rheumatol, 20 (2002), pp. 617-624
[5]
J. Rodriguez-Amado, I. Pelaez-Ballestas, L.H. Sanin, J.A. Esquivel-Valerio, R. Burgos-Vargas, L. Perez-Barbosa, et al.
Epidemiology of rheumatic diseases. A community-based study in urban and rural populations in the state of Nuevo Leon, Mexico.
J Rheumatol, 86 (2011), pp. S9-S14
[6]
J. Alvarez-Nemegyei, I. Pelaez-Ballestas, L.H. Sanin, M.H. Cardiel, A. Ramirez-Angulo, M.V. Goycochea-Robles.
Prevalence of musculoskeletal pain and rheumatic diseases in the southeastern region of Mexico. A COPCORD-based community survey.
J Rheumatol, 86 (2011), pp. S21-S25
[7]
División Técnica de Información Estadística en Salud (DTIES).
Motivos de demanda de consulta externa. Unidad de Investigación, Educación y Políticas de Salud División de Prestaciones Médicas.
IMSS, (2007),
[8]
E. De Pavia-Mota, M.G. Larios-Gonzalez, G. Briseño-Cortes.
Osteoarthritis management in family practice and orthopedia.
Arch Med Fam, 7 (2005), pp. 93-98
[9]
Centro Nacional de Excelencia Tecnológica en Salud.
Metodologíá para la integración de guías de práctica clínica.
Secretaría de Salud, (2007),
[10]
Guidelines for clinical practice: from development to use,
[11]
Atención del paciente con osteoartrosis de cadera y rodilla en el Primer nivel de atención. Guía de Práctica Clínica.
Secretaría de Salud, (2008),
[12]
M.N. DeHaan, J. Guzman, M.T. Bayley, M.J. Bell.
Knee osteoarthritis clinical practice guidelines: how are we doing?.
J Rheumatol, 34 (2007), pp. 2099-2105
[13]
L. Denoeud, B. Mazieres, C. Payen-Champenois, P. Ravaud.
First line treatment of knee osteoarthritis in outpatients in France: adherence to the EULAR 2000 recommendations and factors influencing adherence.
Ann Rheum Dis, 64 (2005), pp. 70-74
[14]
J.N. Pencharz, E. Grigoriadis, G.F. Jansz, C. Bombardier.
A critical appraisal of clinical practice guidelines for the treatment of lower-limb osteoarthritis.
Arthritis Res, 4 (2002), pp. 36-44
[15]
N.E. Walsh, M.V. Hurley.
Evidence based guidelines and current practice for physiotherapy management of knee osteoarthritis.
Musculoskeletal Care, 7 (2009), pp. 45-56
[16]
I.D. Graham, J. Tetroe.
Some theoretical underpinnings of knowledge translation.
Acad Emerg Med, 14 (2007), pp. 936-941
[17]
R.N. Shiffman, J. Dixon, C. Brandt, A. Essaihi, A. Hsiao, G. Michel, et al.
The GuideLine implementability appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation.
BMC Med Inf Decis Mak, 5 (2005), pp. 23
[18]
A.R. Gagliardi, M.C. Brouwers, V.A. Palda, L. Lemieux-Charles, J.M. Grimshaw.
How can we improve guideline use? A conceptual framework of implementability.
Implement Sci, 6 (2011), pp. 26
[19]
G. Guyatt, A.D. Oxman, E.A. Akl, R. Kunz, G. Vist, J. Brozek, et al.
GRADE guidelines: introduction-GRADE evidence profiles and summary of findings tables.
J Clin Epidemiol, 64 (2011), pp. 383-394
[20]
J.P. Forero, A. Muñoz-Urrego.
Manejo de fisiatría y rehabilitación en el tratamiento de la osteoartrosis de cadera, rodilla y mano.
Rev Colomb Reumatol, 9 (2002), pp. 35-40
[21]
Medical management of adults with osteoarthritis [Internet].
Michigan Quality Improvement Consortium, (2005),
[22]
W. Zhang, M. Doherty, N. Arden, B. Bannwarth, J. Bijlsma, K.P. Gunther, et al.
EULAR Standing Committee for International Clinical Studies Including Therapeutics (ESCISIT). EULAR evidence based recommendations for the management of hip osteoarthritis: report of a task force of the EULAR standing committee for international clinical studies including therapeutics (ESCISIT).
Ann Rheum Dis, 64 (2005), pp. 669-681
[23]
National Collaborating Centre for Chronic and Conditions.
Osteoarthritis: national clinical guideline for care and management in adults.
Royal College of Physicians, (2008),
[24]
W. Zhang, R.W. Moskowitz, G. Nuki, S. Abramson, R.D. Altman, N. Arden, et al.
OARSI recommendations for the management of hip and knee osteoarthritis. Part II: OARSI evidence-based, expert consensus guidelines.
Osteoarthritis Cartilage, 16 (2008), pp. 137-162
[25]
O.R.W. De Jong, M. Hopman-Rock, E.C.M. Tak, N.S. Klazinga.
An implementation study of two evidence based exercise and health education programmes for older adults with osteoarthritis of the knee and hip.
Health Educ Res, 19 (2004), pp. 316-325
[26]
P.J. Van der Wees, A.P. Moore, C.M. Powers, A. Stewart, M.W. Nijhuis-van der Sanden, R.A. de Bie.
Development of clinical guidelines in physical therapy: perspective for international collaboration.
Phys Ther, 91 (2011), pp. 1551-1563
[27]
GLIA.
Guideline implementability appraisal [Internet].
Yale University, (2011),
[28]
A.B. Becker, B.A. Israel.
Allen III A. Chapter 3. Strategies and techniques for effective group process in CBPR partnerships.
Methods in community-based participatory research for health, 1st ed., pp. 52-72
[29]
S. Siegel, N.J. Castellan.
Nonparametric statistics for the behavioral sciences international edition.
McGraw-Hill Book Company, (1988), pp. 284-291
[30]
Kappa for nominal data as concordance between multiple raters [Internet]: Department of Obstetrics and Gynaecology. The Chinese University of Hong Kong.
(2011),
[31]
L.J. Cochrane, C.A. Olson, S. Murray, M. Dupuis, T. Tooman, S. Hayes.
Gaps between knowing and doing: understanding and assessing the barriers to optimal health care.
J Contin Educ Health Prof, 27 (2007), pp. 94-102
[32]
R. Grol, J. Dalhuijsen, S. Thomas, C. Veld, G. Rutten, H. Mokkink.
Attributes of clinical guidelines that influence use of guidelines in general practice: observational study.
BMJ, 317 (1998), pp. 858-861
[33]
R. Espinoza-Morales.
Enfoque terapéutico actual de la osteoartritis.
Reumatol Clin, 1 (2005), pp. S8-S15
[34]
M.D. Beaulieu, E. Hudon, D. Roberge, R. Pineault, D. Forte, J. Legare.
Practice guidelines for clinical prevention: do patients, physicians and experts share common ground.
CMAJ, 161 (1999), pp. 519-523
[35]
A. Boivin, J. Green, J. van der Meulen, F. Legare, E. Nolte.
Why consider patients’ preferences. A discourse analysis of clinical practice guideline developers.
Med Care, 47 (2009), pp. 908-915
[36]
S. Michie, M. Johnston.
Changing clinical behaviour by making guidelines specific.
[37]
S. Michie, K. Lester.
Words matter: increasing the implementation of clinical guidelines.
Qual Saf Health Care, 14 (2005), pp. 367-370
[38]
R.N. Shiffman, G. Michel, R.M. Rosenfeld, C. Davidson.
Building better guidelines with BRIDGE-wiz: development and evaluation of a software assistant to promote clarity, transparency, and implementability.
J Am Med Inform Assoc, 19 (2012), pp. 94-101
[39]
D. Van Linthoudt.
Evaluation of the adhesion to the EULAR and OARSI recommendations for the treatment of knee and hip osteoarthritis in current practice.
Praxis (Bern 1994), 98 (2009), pp. 1429-1435
[40]
L. March, B. Amatya, R.H. Osborne, C. Brand.
Developing a minimum standard of care for treating people with osteoarthritis of the hip and knee.
Baillieres Best Pract Res Clin Rheumatol, 24 (2010), pp. 121-145
[41]
H.J. Schunemann, A.D. Oxman, J. Brozek, P. Glasziou, R. Jaeschke, G.E. Vist, et al.
GRADE: grading quality of evidence and strength of recommendations for diagnostic tests and strategies.
[42]
G.H. Guyatt, A.D. Oxman, G.E. Vist, R. Kunz, Y. Falck-Ytter, P. Alonso-Coello, et al.
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
[43]
G.H. Guyatt, A.D. Oxman, R. Kunz, Y. Falck-Ytter, G.E. Vist, A. Liberati, et al.
GRADE: going from evidence to recommendations.
[44]
P. Dieppe, M. Doherty.
Contextualizing osteoarthritis care and the reasons for the gap between evidence and practice.
Clin Geriatr Med, 26 (2010), pp. 419-431
[45]
N. Kashyap, J. Dixon, G. Michel, C. Brandt, R.N. Shiffman.
GLIA. Guideline implementability appraisal v.2.0. Manual.
Yale Center for Medical Informatics, (2011),
Copyright © 2013. Elsevier España, S.L.U.. All rights reserved
Idiomas
Reumatología Clínica

Suscríbase a la newsletter

Opciones de artículo
Herramientas
Material suplementario
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

es en
Política de cookies Cookies policy
Utilizamos cookies propias y de terceros para mejorar nuestros servicios y mostrarle publicidad relacionada con sus preferencias mediante el análisis de sus hábitos de navegación. Si continua navegando, consideramos que acepta su uso. Puede cambiar la configuración u obtener más información aquí. To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here.