Plan for Generating Empirical Support of Psychodrama Efficacy
Rory Remer, PhD, TEP, ABPP
April 13, 2015
Approach
The approach to the project should be guided by four principles.
Simplicity
Sell the program by keeping the procedures and the involvement
Easy
Short
Compelling
KISS it.
Minimize the time and effort required of participants at all levels:
Clients/patients as individuals and group members
Psychodramatists as therapists/facilitators for both group and/or individual settings
Brief (short and simple) outcome measures
Brief (short and simple) demographics and intervention accounts primarily focused on the leaders/facilitators (unless or until members desire/are willing to be more involved).
Proaction
Not the “best.” Not the only (unique). But the first.
Let others have this research as a reference point and focus of discussion, increasing visibility.
Go “viral,” spreading information broadly and fast.
Inclusiveness
“A truly therapeutic procedure cannot have less an objective than the whole of mankind.
But no adequate therapy can be prescribed as long as mankind is not unity in some fashion
and as long as its organization remains unknown. It helped us in the beginning to think,
although we had no definite proof for it, that mankind is a social and organic unity.”
JLM
Who Shall Survive?
pg 3
The aim is to include all psychodramatists, worldwide, in the project—to increase generalizability and visibility (and leverage)
.
Non-Defensiveness
Turning “negatives” into positives.
Welcoming constructive input (Truly, sincerely being able to say, “A valid concern that should be researched further.”).
Transparency—acknowledging strengths, weaknesses, and trade-offs.
Connecting suggestions to possible future more “fine grained” (complex) research efforts. (Experience is Fractal).
Methods
Provided is a general overview of the main aspects of the methods to be employed. More details can and will be provided. However, consistent with the first principle, simplicity is the by-word.
Research Design/Approach
The approach will purposely combine important components of various models, so is not distinctly any one, nor easy to characterize. Aspects of Naturalisitic, Meta-Analytic, Action, Large Sample, Large-scale Survey, Bench-marking, Case/Anecdotal, Evaluation, Outcome, and Feminist models are incorporated.
Research Question
Is Psychodrama, as an intervention modality, demonstrably (empirically) effective?
Participants
Participants are from two levels: providers and recipients.
Service/experience providers. As many psychodramatists (and other therapist/facilitators), worldwide, as are willing to collect data on at least one psychodrama (or other similar) experience.
Service/experience recipients. All clients/patients/students (or similar) who are willing to reflect on the experiences offered by the previously indicated providers and allow their data to be included in the study.
Instruments
Six instruments will be employed: five universal outcome scales (positive/growth impact, symptom reduction, relationship/common factor, positive impact, and threat/negative impact), a psychodrama intervention fidelity checklist, and a demographic questionnaire.
The five universal scales. (See Appendix 1.) Each universal scale is a single dimension/item, applicable to any of the situations/settings in which the providers practice. Each is designed to measure the Gestalt of a participant’s experience in the indicated dimension. They are not exhaustive, but are intended to reflect important and relevant dimensions common to human experiences of change. They scales are self-report measures. The scales are continuous, visual analogues, with gradations marked each .1, in case that degree of precision is desired. Participants indicate their ratings by marking with a vertical line at the points they want to indicate. Reliability, content/face/and construct validity, readability level, and language equivalence for all language versions, will be addressed.
Scale 1: Doing Well
Indicate how positive you are about yourself. Include experiences of feeling: capable, resilient, confident, empowered, adaptable, spontaneous, creative, prepared, knowledgeable, energized, and so forth.
Scale 2: Distress
Indicate how down you are about yourself. Include experiences of feeling: uncertain, unprepared, guilty, defensive, ashamed, and so forth.
Scale 3: Connectedness
Indicate how close you feel to others in your life. Include experiences of feeling: accepted, connected, supported, belonging, understood, and so forth.
Scale 4: Helpfulness
Indicate how helpful the experience/intervention has been for you. Include reactions of feeling: encouraged, supported, better prepared, more knowledgeable, assured, and so forth.
Scale 5: Harm
Indicate how much harm/hurt the experience/intervention has done you. Include experiences of feeling: wounded, threatened, overwhelmed, caught off guard, coerced, and so forth.
Intervention Fidelity Checklist. (See Appendix 2.) A brief, basic checklist indicating the presence (or absence) of defining elements of psychodramatic intervention, derived from the Hollander Psychodrama Curve (Hollander, 1969). Will be used to categorize an intervention as psychodramatic. (Intended to provide a structure comparable to treatment manuals demanded for Empirically Supported Interventions [ESI] and/or Evidence Based Practice [EBP].)
Demographic questionnaire. (See Appendix 3.) Designed to obtain general background information for description and/or categorization of participants.
Operational Definitions
Operational definitions are directly and simply related to the instruments employed.
Positive outcome. Change (pre-post difference) score on the Doing Well (DW) Scale.
Symptom reduction. Change (pre-post difference) score on the Distress (D) Scale.
Relationship strength. Change (pre-post difference) score on the Connectedness (C) Scale.
Helpful impact. Change (pre-post difference) score on the Helpfulness (He) scale.
Negative impact. Change (pre-post difference) score on the Harm (Ha) scale.
Psychodramatic intervention. All items in the “Component” section of the Intervention Fidelity Checklist checked as being present during the intervention.
Intervention. All the experiences/interactions involving the providers and recipients from the initiation of the session until the closing of the session.
Research and Statistical Hypotheses
Four research and statistical hypotheses will be tested, reflecting the Research Question/Problem.
H1. Psychodramatic intervention will significantly impact positive change. The mean change score on the Doing Well (DW) scale will be significantly positive.
H1: µDW > 0
The effect size for change scores on the Doing Well (DW) scale will be positive and practically significant.
H2. Psychodramatic intervention will significantly impact symptom reduction. The mean change score on the Distress (D) scale will be significantly negative.
H1: µD < 0
The effect size for change scores on the Distress (D) scale will be negative and practically significant.
H3. Psychodramatic intervention will significantly impact connectedness. The mean change score on the Connectedness (C) scale will be significantly positive.
H1: µC > 0
The effect size for change scores on the Connectedness (C) scale will be positive and practically significant.
H4. Psychodramatic intervention will have a significant positive impact. The mean post-test score on the Helpfulness (He) scale will be significant.
H1: µHe > 0
The effect size for post-intervention scores on the Helpfulness (He) scale will be positive and practically significant.
H5. Psychodramatic intervention will not have a significant negative impact. The mean post-test score on the Harm (Ha) scale will not be significant.
H1: µHa > 0
The effect size for post-intervention scores on the Harm (Ha) scale will be negligible and practically insignificant
Procedures
Psychodramatists and other facilitator (service providers) will be contacted via email, postings to newsletters, and snowball-sampling to elicit participation in the study. The purpose, procedures, and requirements of the study will be explained. Informed Consent documents will be presented and explained. Only those participants willing to comply with the study protocol for the initial evaluation will do so, and no names or ID #’s will be included. For those willing to participate in a longitudinal follow-up, and/or other studies, space will included to allow for providing a name and contact information, or an ID#, if one has been previously assigned. Once an ID# has been assigned, the identifying information (name and contact data) will be removed from the data and filed separately, to ensure confidentiality.
Provider-participants will be asked to use the three scales pre and four scales post (negative impact scale being added to the post intervention assessment) for at least one session they conduct, to complete the Intervention Fidelity Checklist and the Demographic Questionnaire. (See announcement and informed consent for providers, Appendix 4.) They will be instructed in soliciting the permission of their service recipients’ data using the script the informed consent for recipients, Appendix 4.)
Recipient-participants will be asked to use the three scales pre and five scales post (positive [Helpfulness] and negative impact [Harm] scale being added to the post intervention assessment) for evaluating the chosen experience, and submit the anonymous demographic questionnaire.
Sessions will be conducted as they typically would be. After a session, recipient-participants will be offered the opportunity to submit their pre-post ratings on the four scales and the demographic questionnaire. Submission of the data will constitute recipient participant informed consent. These participants will also be offered the opportunity to be assigned an ID#, if they would be interested and willing to be involved further (e.g., other studies—longitudinal follow-up, evaluation of more specific interventions.)
Data from both level participants will be sent to a central data collection repository for entry, collation, and analysis.
The integration of the use of the outcome scales pre-post all sessions, to become part of the intervention process, will be one aim of the study. Doing so would allow a conscious and continual focus on intervention impact to become part of the psychodrama culture, provide a consistent orientation (warm-up) to sessions, and address one threat to internal validity (testing) of this and other research employing the scales as measures. (And also serve as a model for other therapeutic modalities.)
Analyses
Analyses are in three stages: Outcome Scales Psychometric Properties, Preliminary Data Analysis, and Primary Study Analyses.
Outcome scales psychometric properties. The scale data will be analyzed for language-version equivalence, readability, content/face/construct validity, and reliability.
Language-version equivalence. After translation/back-translation protocol for accurately rendering instruments in another language have been implemented, the various language versions of the outcome scales will be analyzed for differential item functioning using Item Response Theory (IRT) analysis. The three parameter logistic model (IRT 3PL, Graded Response, Non-Rasch for Ordered Polytomous Data, de Ayala, 2009, p. 222) will be used to fit the data, to take into account the adjustment for possible social desirability influence. Different language versions will be linked/equated to ensure comparable metrics and converted to T-score scales to facilitate interpretation of results.
Readability. A readability analysis will be done to ensure scales can be adequately understood by participants. Readability level will be calculated and reported for each version and overall.
Content/face/construct validity. Scales will be submitted to three member expert panels for each language version to assess the validity of the scales for measuring the purported constructs. Inter-rater agreement will be calculated and reported for each version and overall.
Reliability. Appling the IRT 3PL(Graded Response, Non-Rasch for Ordered Polytomous Data, de Ayala, 2009, p. 222) Model, each scale’s reliability will be calculated and reported for each version and overall.
Preliminary Data Analysis. Summary statistics will be produced for each scale. Data will be analyzed for skewness, kurtosis, univariate and multivariate out-liers, sphericity and so forth, to ensure that the data meet the assumptions for the primary analyses. Required adjustments will be implemented (e.g., data transformations, elimination of outliers). Missing data, if any, will be addressed by using the IRT approach.
Primary Study Analyses. Three levels of primary analyses will be applied to the data to address the hypotheses stated previously—multivariate analysis, univariate analysis by scale, and meta-analysis by scale and overall.
Multivariate analysis. A Hotelling’s-T will calculated for the vector of scale mean change scores.
Univariate analysis. Univariate-t’s will calculated for the each scale mean change score.
Meta-analyses. The effect sizes will be calculated for each scale language version. These data will be subjected to meta-analytic techniques combining across scales and language versions.
Bench-marking analyses. The effect sizes will be calculated for each scale language version. These data will be subjected to meta-analytic techniques combining across scales and language versions and bench-marked with already Evidence based (perhaps empirically supported) interventions (e.g., CBT [cognitive behavioral therapy], DBT [dialectic behavioral therapy])
Appendix 1: Outcome Scales
Hay Outcome Scales
How Are You (HAY) Scales
Pre-Session
Scale 1: Doing Well
Indicate how positive you are about yourself. Include experiences of feeling: capable, resilient, confident, empowered, adaptable, spontaneous, creative, prepared, knowledgeable, energized, and so forth.
My sense of well I am doing is:
0…..….1…..….2…..….3…..….4…..….5…..….6…..….7…..….8…..….9…..….10
Scale 2: Distress
Indicate how down you are about yourself. Include experiences of feeling: uncertain, unprepared, guilty, defensive, ashamed, and so forth.
My degree of distress is:
0…..….1…..….2…..….3…..….4…..….5…..….6…..….7…..….8…..….9…..….10
Scale 3: Connectedness
Indicate how close you feel to others in your life. Include experiences of feeling: accepted, connected, supported, belonging, understood, and so forth.
My sense of being connected is:
0…..….1…..….2…..….3…..….4…..….5…..….6…..….7…..….8…..….9…..….10
Post-Session
Scale 1: Doing Well
Indicate how positive you are about yourself. Include experiences of feeling: capable, resilient, confident, empowered, adaptable, spontaneous, creative, prepared, knowledgeable, energized, and so forth.
My sense of well I am doing is:
0…..….1…..….2…..….3…..….4…..….5…..….6…..….7…..….8…..….9…..….10
Scale 2: Distress
Indicate how down you are about yourself. Include experiences of feeling: uncertain, unprepared, guilty, defensive, ashamed, and so forth.
My degree of distress is:
0…..….1…..….2…..….3…..….4…..….5…..….6…..….7…..….8…..….9…..….10
Scale 3: Connectedness
Indicate how close you feel to others in your life. Include experiences of feeling: accepted, connected, supported, belonging, understood, and so forth.
My sense of being connected is:
0…..….1…..….2…..….3…..….4…..….5…..….6…..….7…..….8…..….9…..….10
Scale 4: Helpfulness
Indicate how helpful the experience/intervention has been for you. Include reactions of feeling: encouraged, supported, better prepared, more knowledgeable, assured, and so forth.
My sense of being helped is:
0…..….1…..….2…..….3…..….4…..….5…..….6…..….7…..….8…..….9…..….10
Scale 5: Harm
Indicate how much harm/hurt the experience/intervention has done you. Include experiences of feeling: wounded, threatened, overwhelmed, caught off guard, coerced, and so forth.
My sense of being harmed is:
0…..….1…..….2…..….3…..….4…..….5…..….6…..….7…..….8…..….9…..….10
Appendix 2: Intervention Fidelity Checklist
Check if the following elements/aspects of psychodramatic intervention were present.
Components
Warm-up
Theme/Issue Selection
Scene Setting
Catharsis of Abreaction
Surplus Reality
Catharsis of Integration
Sharing
Roles
Protagonist
Auxiliary Ego
Double
Audience
Techniques
Role-reversal
Mirror
Concretization
Session Data
Date: Title:
Type of Session: Group Individual Couple Family
Therapy Training Growth Supervision Other (please specify)
Classical Psychodrama Situational Psychodrama Sociodrama Other (please specify)
Length of Session: Number of Participants: Participants’ Age Range:
Facilitator (Provider) Data
Name or ID# of Facilitator: Years of Professional Experience:
Certification: TEP CP Other (please specify) Years of Psychodrama Experience:
Number of Psychodrama Training Hours: Highest Degree Earned:
Appendix 3: Demographic Questionnaire
Please respond the following questions about yourself.
What is your current age in years? ____________
What is your primary language? ____________
What is your race or ethnicity? ___________
What is your religious affiliation? ____________
What is your primary country of residence? ____________
Please describe your gender (by circling your choice).
- Female
- Male
- Transgender
- Other ____________________
Please identify your sexual orientation (by circling your choice).
- Exclusively Heterosexual
- Somewhat Heterosexual/Somewhat Homosexual
- Exclusively Homosexual
If you have mental health diagnosis and are willing to share it, please share it/them here:
Diagnosis/diagnoses
How many mental health/personal growth experiences have you had?
How many psychodramatic experiences have you had?
ID# (if already assigned):
Appendix 4: Informed Consents
(These IC’s are being reviewed, revised, and vetted—so are only initial versions)
Informed Consent: Experience Recipient
Title: Psychodrama Efficacy
Explanation and Purpose of Research
You are being asked to participate in a research study of the efficacy of psychodramatic interventions. You have been asked to participate in this study because you have chosen to participate in a psychodrama-like experience or one with which psychodrama-like experiences might be compared.
Description of Procedures
As a recipient-participant in this study you will be asked to indicate your reactions to your experience both prior to and after the experience. You will be asked to indicate your reactions by marking your ratings on four continua (visual analogue scales). You simply put a vertical line on the scale at the point that indicates you reaction. Three ratings are requested prior to your experience and four after it. You will also be asked a few demographic questions (such as race/ethnicity, age, and primary language). Total time involved should be about three minutes.
The researchers appreciate your time and commitment to this evaluation. Please recognize participation in this study is voluntary and you may withdraw from the study at any time by not submitting your reaction ratings and demographic questionnaire. The researchers encourage you to complete all items, but you are free to skip those that answering make you uncomfortable.
Participation and Benefits
Your involvement in this study is completely voluntary and you may withdraw from the study at any time, as indicated previously. Besides contributing to the evaluation of psychodrama efficacy, which may allow provision of and reimbursement for such services, reflecting on your psychological state by rating your reactions may enhance the benefits of your chosen experience.
Questions Regarding the Study
If you have any questions about the research study, please address them to the person(s) providing your experience, or you may contact the researchers. (Supply email address).
Potential Risks
The only possible risk, though unlikely, in this study is discomfort with the questions you are asked. If you experience psychological or emotional discomfort during the rating process, you may stop answering questions at any time and not submit the materials. All other aspects of the experience will be the same as if you choose not to participate. Any discomfort or problems with the experience in which you will be participating should be addressed to the facilitator(s) of that experience.
Unless you elect to participate beyond the arrangement already addressed, you should not be subject to any other risks, since data will not be linked to any identifying information. However, you may choose to participate further by being involved in follow-up assessments of your experience or other similar experiences. In these cases, you will be offered the option of being assigned an ID# to be associated with your data, and asked to provide contact information. Once an ID# is assigned, it will be associated with your data. The ID# and associated contact information will be kept separate from the research data and protected.
Potential risks related to your extended participation in the study may include loss of confidentiality, though unlikely. Confidentiality, as indicated, will be protected to the extent that is allowed by law.
Consent to Participate
Note: Your consent to participate in the basic/limited study is indicated by your submitting the seven scale ratings (three pre-experience and four post-experience) and the demographic questionnaire.
For extended participation, to indicate that you accept these terms, that you have read and understood the above statements, and that you give your informed consent to participate, please select “Yes” on either or both statements below.
If you are interested and willing to be contacted for a future evaluation of this experience, please circle
Yes
And provide your Name and Contact Address here (an ID# will be assigned you for anonymity).
Name (or ID# if assigned):
The following information is not needed if you already have been assigned and ID#.
Signature: Date:
Contact Information:
If you are interested and willing to be contacted for other experiences, please circle
Yes
And provide your information above (if you haven’t already).
Informed Consent: Experience Provider
Title: Psychodrama Efficacy
Explanation and Purpose of Research
You are being asked to participate in a research study of the efficacy of psychodramatic interventions. You have been asked to participate in this study because you have chosen to participate in a psychodrama-like experience or one with which psychodrama-like experiences might be compared.
Description of Procedures
As a provider-participant in this study you will be asked to familiarize yourself with the study materials (informed consent, outcome scales, and demographic questionnaire), ask prospective recipient-participants about their willingness to participate, supply them with the informed consent document and other study materials, and collect the materials from those who choose to participate at the end of the experience you provide. In some cases, due to the readability level of the instruments, you may be asked to read the scales to the recipient-participants. However, you are not expected or permitted to interpret or help them generate their responses. The session/experience you provide will not in any other way be different from what you would do if you weren’t participating in the study.
After providing the experience you will be asked to complete the Intervention Fidelity Checklist, the Session Data form, the Facilitator (Provider) Data form, and the demographic questionnaire. You will also be asked to send all materials, including the Experience Provider Informed Consent, to the researchers.
Total time involved will depend on the number of recipient participants, but should not be more than one hour for a particular experience provided.
The researchers appreciate your time and commitment to this assessment. Please recognize participation in this study is voluntary and you may withdraw from the study at any time by not submitting your materials. The researchers encourage you to complete all items, but you are free to skip those that answering make you uncomfortable.
Note: Your consent to participate is indicated by your submitting the study materials to the researchers, unless you opt for further participation (collecting information on other experiences you provide). In which case you will be assigned an ID# for aggregating data.
Potential Risks
No risks are anticipated. All aspects of the experience will be the same as if you choose not to participate. Any discomfort or problems with the experience that you will be providing should be minimal for you and the recipients beyond those you would anticipate from providing the experience outside the study. You should be prepared to handle situations that arise as you generally would, although in the unlikely event of a reaction to rating scale wording, you may have to deal with that reaction.
Unless you elect to participate beyond the arrangement already addressed, you should not be subject to any other risks, since data will not be linked to any identifying information. However, you may choose to participate further by being involved in follow-up assessments of other similar experiences you provide. In these cases, you will be offered the option of being assigned an ID# to be associated with your data, and asked to provide contact information. Once an ID# is assigned, it will be associated with your data. The ID# and associated contact information will be kept separate from the research data and protected.
Potential risks related to your extended participation in the study may include loss of confidentiality, though unlikely. Confidentiality, as indicated, will be protected to the extent that is allowed by law.
Participation and Benefits
Your involvement in this study is completely voluntary and you may withdraw from the study at any time, as indicated previously. Besides your contributing to the evaluation of psychodrama efficacy, which may allow provision of and reimbursement for such services, receiving feedback on the experiences you provide may help you be more effective in providing similar experiences in the future and establish better working relationships with your recipients.
Questions Regarding the Study
If you have any questions about the research study, you may contact the researchers. (Supply email address).
Consent to Participate
Note: Your consent to participate in the basic/limited study is indicated by your submitting the all study materials to the researchers.
For extended participation, to indicate that you accept these terms, that you have read and understood the above statements, and that you give your informed consent to participate, please select “Yes” on either or both statements below.
If you are interested and willing to be contacted for evaluating other experiences, please circle
Yes
And provide your Name and Contact Address here (an ID# will be assigned you for anonymity).
Name (or ID# if assigned):
The following information is not needed if you already have been assigned and ID#.
Signature: Date:
Contact Information:
Selected References
de Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: Guilford.
Samejima, F. (1973). Homogeneous case of the continuous response model. Psychometrika, 38, 203-219.
Hollander, C. E. (1969). A process for psychodrama training: The Hollander psychodrama curve. Denver, CO: Snow Lion.
