Career Profile: Program Evaluation
The Career Profiles in Linguistics section regularly highlights career paths taken by linguists. If you would like to recommend someone (including yourself) for a future profile, please contact Career Linguist.
I recently had the opportunity to chat with Julie Solomon, Ph.D., the Founder and Principal of J. Solomon Consulting, LLC, a program evaluation consulting firm that focuses primarily on the health sector.
Her path: According to her bio: Prior to founding the company in 2008, she was a Senior Research Associate at an applied social science research firm in Los Altos, California that specialized in health and social issues.
While this context makes her leap to start her own business seem very linear and logical, this way of storying her experience doesn’t quite capture some of the really fascinating twists and turns. I think Julie must have been reading Katharine Brooks, because the professional journey map that she shared with me looks very much like a “wise wanderings map.”
In a nutshell, Julie has spent the past 16 years directing mixed-method (quantitative/qualitative) program evaluation and applied research projects and providing program planning and evaluation consultation and training services to non-profit organizations, government agencies, coalitions, and grantmakers.
But what is program evaluation? Evaluation involves the systematic collection, analysis, and interpretation of data to answer questions about programs, projects, policies, personnel, products, or organizations, including whether they are achieving their objectives.
The work of program evaluation might involve asking questions like:
- Is the program delivering the services it intended to deliver?
- Is the program reaching the intended target population?
- How do program stakeholders feel about the program? What suggestions do they have for improvement?
- Is the program achieving the intended outcomes (e.g., knowledge, skills, attitudes, behaviors, health status)
- How do the program’s costs and benefits compare?
And you would be doing this in order to:
- Inform program improvement
- Provide accountability to funders, Boards, and other stakeholders
- Secure support from new partners/stakeholders
- Inform the field about best practices
- Influence policy
Evaluators work in many professional sectors, and Julie’s principal topical areas of focus have included youth sexual and reproductive health, HIV/AIDS, maternal and child health, and development of the health care workforce. She works both in the U.S. and internationally, and has conducted evaluation study fieldwork in Belize, Colombia, Honduras, Guatemala, Mexico, and Panama in the past several years.
Can you give an example? One project that Julie recently worked on was a retrospective evaluation of the GOJoven: Youth Leadership in Sexual and Reproductive Health Program in Belize, Guatemala, Honduras, and Mexico (http://www.gojoven.org). The project included the following data collection activities:
- Desk review
- Online survey of program participants and their organizations
- Fieldwork in-country
- Focus groups
- On-site interviews
- Participant-observation: national program meetings
- Most Significant Change (MSC) stories and discussions
The work culminated in a National program meeting and MSC work with GOJoven in Guatemala, 2012.
Could you give us one more program example? A program that Julie is currently working with is Bridge to Employment funded by Johnson & Johnson Corporate Contributions, managed by FHI 360. Julie serves as the external evaluator for several sites. Check out their website (http://www.bridge2employment.org/ ) to learn more about the work that Bridge to Employment is engaged in.
What academic training do you need to get into program evaluation? According to Julie, many evaluators have backgrounds in areas such as Public Health, Education, Social Sciences, Public Policy, or Environmental Studies, and one typically must hold at least a Master’s Degree in order to be a project director.
Julie herself holds Ph.D. and A.M. degrees in linguistics from Stanford University and a B.A. in linguistic anthropology and Spanish literature from Brandeis University. Her dissertation focused on phonological and syntactic variation in the Spanish spoken in Valladolid, Yucatán, Mexico. In addition to being fluent in English and Spanish, she also speaks basic French, Portuguese, Italian, and Yucatec Maya.
What is it about your training as a linguist that makes you particularly effective as a program evaluator? Linguists bring three core competencies:
- Effective cross-cultural communication
- Awareness of language processing challenges
- Emic and etic perspectives
Thinking on the macro-level, these abilities help us to shift our perspective to be aware of things like: What is salient to program implementers, participants, funders, and other stakeholders (emic)? What is salient to the evaluator (outsider), based on his or her expertise (etic)? How can both perspectives add value to the task at hand (planning, analysis, interpretation, etc.)?
Jumping to the micro interactional level, we are likely to be more aware of things like the construction of questions and how that might be interfering with comprehension, for example.
Our ability to conceptualize an analysis bottom-up to top-down (and vice-versa), also help an evaluator to think about impacts: At the level of the individual, the family, the organization, nationally, and internationally.
Interests in teaching?
Also, something that people might not be thinking about is that for people who enjoy teaching, there are opportunities to facilitate training workshops with staff of client organizations, students, and even other evaluators. There are always opportunities to mentor or advise young or new evaluators. So, if you are an educator, you might be happily surprised to find that there are many ways to find expression of this interest in contexts outside of the traditional higher-education classroom.
What is your advice for someone trying to break into the field of program evaluation?
Conduct informational interviews! In Julie’s case, she got started by talking with people in international development, applied social science research, evaluation, and consulting.
Join the American Evaluation Association (http://www.eval.org)
Get as much experience as you can with: Participant observation, Focus groups, Survey design and analysis
- Take classes in a range of research methods
- Learn to use data analysis software
- Take classes in topical areas of interest
- Look for relevant internships and part-time RA positions
- Read articles in evaluation-focused journals
- Attend evaluation conferences and pre-conference workshops (student discounts usually available)
Thank you Julie for this edifying look at this fascinating world of work!
Return to Career Profiles in Linguistics.