Archives

News

Cold Case: Having to Construct an Evaluation after the Fact

Cold case detectives aren’t just on TV. Some of them are also called evaluators, experts called in to help a project complete an outcome evaluation after a program has been designed and implemented. In the worst situation,  a cold case evaluator is called in to complete an evaluation with no data or bad data. Frequently, time is short, a funding source is demanding a final evaluation report, and program staff are disinterested and maybe even antagonistic about having an evaluator look at their outcomes.

As a consultant who has been in this situation more than once, I have this to say: You would be amazed at what passes for data collection in many programs – hand-signed attendance sheets, ginned-up pre and post tests, and anecdotes galore. Interesting material, often, but not the stuff of decent evaluations.

What to do when you’re asked to evaluate a program that is nearing the end of its funding period and has had no solid evaluation system put into place? Here are some ideas gleaned from my own experience as a cold case evaluator.

#1: Enlist program staff in your cause.

A quick way to guarantee that you will never get any data with which to evaluate the program is to alienate the program staff. If they feel you are judging them or taking a superior attitude because you’re in the evaluator position, they will make your job harder. Instead of tsk-tsking your way around, make program staff your partners in telling the program’s story in the most accurate way possible.

#2: Use what you have.

Is there any program data? Separate the wheat from the chaff and use it. Are program participants still engaged? Develop a retrospective survey instrument to gather their insights about program impact. Is there a staff person who has been involved with the program from the beginning? Ask her/him a thousand questions. You may find out there’s more data laying around than anyone knew. They didn’t tell you because they didn’t think it was important. Moreover, an evaluation encumbered by lack of decent data can be greatly enhanced by attention to good process evaluation. In that case, telling the program’s story through the views of informed observers can also give insight into the difficulty in establishing an outcome evaluation.

#3: Create a beautiful product.

Present whatever data you have in a clear, readable format. Use graphs and charts whenever you can. Compare the program’s results to the results of other similar programs. Bulk up the content with the insights of program staff and vignettes about representative participants. Include a carefully crafted and objectively stated list of ‘areas to consider for further development.’ In this list, be sure to include the need to design the outcome evaluation when the program is designed and to establish good data collection protocols from the beginning. Say this as a going forward recommendation, not as a criticism. By now, program staff know they missed the boat on designing an outcome evaluation, no need to rub it in. Last, make sure the evaluation report looks good. I work with a professional graphic designer on all my products; it’s money well-spent.

There are important things to be learned from every program’s implementation. Sometimes, we can’t measure all of them but often we can know more than we think if we are patient, professional, and persistent, just like a good cold case detective.

 


Failure IS an Option

How many times have you heard someone declare, “Failure is not an option?”

Sure it is. It has to be.

What passes for a cheerleading slogan shouldn’t be advice for living or, more to today’s point, advice for a successful professional life. To be successful means to fail because it is in the owning up to failure and the determination to do better that the fine edge of one’s career is honed.

My most admired colleagues are those who have made big mistakes and suffered a lot of negative public reaction, even humiliation, and then decided to hunker down, work hard, stay true to themselves, and be successful. It’s a great thing to see. They don’t deny the failure, they own it. But they don’t dwell on the failure, they sort through what happened and why, use what will help them develop professionally, and bury the rest.

They don’t blame other people, whine and complain, feel sorry for themselves, or hide from the world.

Owning one’s own failure is the number one way to turning failure into fuel for success. When you own it, you are saying that the analysis of whose fault was whose is over. You are taking responsibility and from there on out, you will be in charge of how the failure will influence your professional demeanor and decision-making.

When people talk about ‘seasoned professionals,” this is what they’re talking about – people who got to the top of their game the hard way.

 


Do You REALLY Want Consumer Participation? Five Ways to Make It Happen

 

The lack of consumer participation is very high on the group hand-wringing index. Everyone knows they should have consumers involved in planning and decision-making but it’s just so ‘gosh-darn’ hard. No argument there, consumer participation is very difficult but it can be done and done well. It just takes time, commitment, and some serious, sustained creativity.

Here are five ways to increase the involvement of consumers in your planning and decision-making:

#1: Take the time to establish consumer participation as a shared value in your group.

It’s easy to announce an effort to involve consumers; easier still to say that the funding source requires consumers to be at the table. It’s one more box to be checked off. However, commitment to consumer participation among group members will be weak if there isn’t an informed and solid consensus about its value. How to accomplish this? Form a study group to examine different models of involvement. Talk to consumers about what it would take to get their enthusiastic involvement. Honestly assess barriers that may exist to participation and gauge the group’s willingness to overcome those barriers. This might mean changing the way business is done. Is the group willing to do that?

 #2: Determine what you want from consumer participation.

If you only want to fulfill a funder requirement or your own vague sense of the need for inclusion, consumers will see right through you. No one wants to attend meetings or participate in functions where they are just filling a chair with a big sign on it – CONSUMER. They want to have a genuine function and be important to the group’s planning, decision-making and community impact. One way to address this is to hammer out a consumer participation job description. Best to do this with consumers at the table to avoid the inevitable mistakes that will occur if the professionals go it alone.

#3: Be prepared to change how you do business.

First and foremost on this list is the need to look at WHEN meetings are held. It is generally not going to be possible to involve consumers in meetings that are held during the business day. Bottom line: many consumers are working and their work does not afford them the opportunity to attend meetings during the day. Conversely, professional folks are basically paid to attend meetings. That’s what they do during the day. At night and on weekends, they have family responsibilities, recreation, maybe education to fill their time. Are you willing to hold meetings in the evening or on weekend mornings? A big shock to the group system, that’s for sure, but important to think about. One way to address this may be to alternate weekday meetings with meetings held in the evenings or weekends.

#4: Find a way to pay consumers.

So often, every member of a coalition is being paid to attend meetings but we expect consumers to VOLUNTEER. Consumers quickly discern that they are the only ones expected to show their altruistic selves and it engenders resentment. It is not easy to find funding to support paying consumer stipends because not all funders understand the value of consumers’ time or buy into the notion that they should be paid for their participation. This is where a strong shared commitment to consumer participation can convince funders of its value. Paying people also speaks to their value in the process. If you really want consumer participation, show that you value it. One way to address this might be to levy a membership fee to support coalition operation as well as the payment of stipends. Seeking a sustainable line item in grant requests is another possible strategy.

#5: Make your meetings worth attending.

Don’t go through all the difficulty of getting good consumer participation only to have meetings filled with long, boring committee reports and extended ‘insider’ discussions. Make your meetings accessible in terms of location, content, and ambiance. Welcome all members. Create agendas that are decision-focused. Share useful information. Provide training on new skills. And make the environment warm, inviting, and engaging. If I had my way, every meeting would start off with a big pot of soup or table full of everyone’s favorite casseroles. Then when Consumer X walks into the school cafeteria for an after-work meeting, he smells dinner, he sees happy people, he’s nourished and ready to work. How you do this is by your group’s members making a commitment to making it happen. It’s not rocket science, it’s just chicken and rice.

Consumer participation will make whatever you’re doing better and more effective. But you get what you pay for – in terms of effort, commitment, and resources. Think about it – is it worth it to you?

Always interested in your comment and ideas. Let me know what you think!


Does Milwaukee Need SDC?

What’s next for SDC (Social Development Commission) was the question of the day in this morning’s Milwaukee Journal Sentinel. Community leaders, probably several  who haven’t graced the doors of SDC for ten years, chimed in with their predictions. What they know about SDC is what they have read in the paper, and while the MJS has been a steady observer of SDC for many years, its coverage (intentionally or not) has been focused on a slice of life in the agency. That said, clearly SDC has troubles of significant proportion, sufficient to call the question.

SDC was established by Mayor Frank Zeidler in 1963 as a quasi-governmental entity to study the problems of poverty and racism and recommend solutions. In 1964, SDC was designated as Milwaukee County’s Community Action Agency as part of the federal War on Poverty. The agency’s enabling legislation includes city and county ordinances and state statute; its legally defined mission is to address the problems of poverty in this community.

Does Milwaukee need SDC? My response is YES, IF…..

The appointing authorities identified in state statute and city and county ordinances for the statutorily-created Social Development Commission appoint people of substance and stature to the Board of Commissioners;

Those appointees attend every board meeting, serve on appropriate committees, and give their best effort to the governance of the agency;

Board members act with courtesy, study the issues before them, and deliberate with the best interests of low-income people as their top priority;

A highly competent executive management team is quickly installed;

That management team includes professionals with excellent management skills, proven leadership ability, and demonstrated commitment to the unique empowerment role of community action in this country;

The board of commissioners and the management team invest its time and resources in a fast-track, in-depth diagnostic and strategic planning process, using its own resources and expertise and calling on local planning resources for assistance.

The agency re-commits to the fundamental principles of community empowerment, elimination of poverty, and creation of opportunity for everyone;

Elected officials and those in positions of power avoid backroom deals, refuse to foster hostile takeovers by other agencies, and fully support the efforts of the agency to right itself.

Does Milwaukee need SDC? There is just one answer for that. Yes.

The poverty rate in this city is 29.4%.

Managed well, with leadership that is enthusiastic and inclusive, with a plan that reaches every part of the community, respects every person, and organizes every resource, with a board that is careful and collaborative and forward thinking and with a community that looks for progress instead of rejoicing in mistakes, SDC can change that number. Now is the time to give the agency just that chance.

_______________________

Disclosure: I worked at SDC for several years, starting as an intern in 1976, and most recently, as Director of Planning, Research and Community Involvement from 1990 to 1995; I also served a term as an elected SDC Commissioner. Recently, I worked as a consultant for the agency with major responsibility for writing the agency’s Head Start application.


Seeing is Believing

When evaluating a program or service, nothing beats a site visit.  Yes, it’s important to review the numbers, look at the logic model, quantify outcomes, and gather customer/client satisfaction data.  These fundamental sources of information are essential to painting the evaluation picture.  But the heart and soul of an evaluation comes from face to face meetings, observations, and ‘walking around’ a program.

I will be doing three site visits in September – three very different agencies in very different parts of Wisconsin, requiring a lot of travel and a lot of time.  So why not just interview people over the phone or do a ‘Go To Meeting’ virtual meeting?

Here’s the answer:  I can’t tell if there’s a ‘there’ there unless I go see.  Seriously, the ability of executive directors to describe their programs in glowing terms is legendary.  If so inclined, an enthusiastic executive director can turn tens of participants into hundreds, good outcomes into astonishing accomplishments, well, you get the idea.  If I’m evaluating a program, I need to make sure the program is operating as described, the participants are really present and engaged and the outcomes are legitimate.

In my experience, these are the things that make for a great site visit:

1.  Genuine welcome:   This begins at the front door.  Do people know I’m coming?  Are they gracious and friendly?  Are the people I need to see available?  Does it appear that the evaluation site visit is a priority?

2.  Openness:  Do people appear to be sharing information freely?  Or are they guarded in what they share?  Does everyone in a group discussion speak or just the executive director?  Are people nervous about sharing or eager to tell their story?

3. Confidence and pride: Are people proud of their organization and happy to tell their story?  Are they willing to share war stories, to describe barriers or problems encountered and how they were overcome?

4. Inclusiveness: Does the executive director leap up to go find “Mary” who is the expert in a particular area or call in a client waiting at the front desk to relate his experience with the program?  In other words, does the executive director or program staff want to include others in explaining the program? 

5. Real Deal Feel:  When I leave, do I feel like I saw the real deal or a show staged for my benefit?  There’s no way to quantify this, but an experienced evaluator can sense an artificiality in the site visit that lets her/him know that the real program wasn’t shared (and may not actually exist).

These are the things I’ll be looking for in September as I travel around Wisconsin.  What about you?  Done evaluation site visits?  Been site visited?  What have been your experiences?  What can we learn from you?


How Many Homeless People on the Head of a Pin?

The current controversy about Wisconsin’s job numbers brings to mind the longstanding debate about how to count people who are homeless.  For several years, the federal government insisted that people are homeless only if they are living in a shelter or transitional housing or on the street, in a car, or other place not fit for human habitation. 

So you’re 22 years old.  The apartment building where you had been living with three other guys was condemned by the city because of hundreds of code violations and you had to leave your apartment.  For a couple of days, you stayed with somebody you met at a bar but he’s telling you to get out by the end of the week.  Until very recently, the federal government would not consider you to be homeless and you would not be included in the regular census (Point in Time Count) of homeless people that every community is required to conduct in order to receive federal homeless funding.

Those of us who thought the homeless count should include families doubling up, youth who were couch-surfing (moving from place to place every night), and others in precarious and dangerous housing situations railed against the fed’s restrictive definition.  Each time, Milwaukee did a Point in Time count, we would be careful to add that there were many more homeless people than reflected in the actual counted number.  For example, in January of 2011, the local census counted 1,466 homeless people.  We figured the real number was at least three times larger.

Still, the homeless count measure was reliable across time (year to year) and across sites.  Cincinnati, Nashville, San Francisco, Denver, Milwaukee all counted homeless people the same way using the same definition.  This made the Point in Time a valid measure.  Try as we might to counter the official Point in Time with other measures, we had to own up to the fact that the Point in Time was the federal government’s official count of people who were homeless.  This was the count reported to the U.S. Congress.  Like it or not, it was the accepted measure.

The same is true with the U.S. Bureau of Labor Statistics annual employment report.  This is the accepted measure for unemployment.  Flawed in its methodology perhaps (just like the Point in Time) but consistent over time and across sites.  Pennsylvania, Arkansas, Nebraska, Oregon, and Wisconsin employment data all calculated the same.  Sure, there are a lot of ways to calculate employment/unemployment but until the federal government changes the measure, that’s the standard we all have to live with whether we’re running for office or just writing a report.

__________________
A copy of Milwaukee’s 2011 Point in Time report can be found at www.milwaukeecoc.org.

 

 


Employers and CCAP: What role do companies play in creating worker shortage?

Governor Walker’s announcement of the plan to use WHEDA’s bonding authority to raise $100 million to invest in Milwaukee’s 30th Street Industrial Corridor is welcome news in many ways.  Most meaningful in the short term is that this economically depleted neighborhood might become the battleground for the race for Governor.  If only for the next month, the issues of this once booming part of town could be on the front page.  The announcement also demonstrates that there are a lot of ways to tackle economic development.  Using the WHEDA bonding authority as an instrument for economic development rather than continuing the non-job creation strategy of more and more housing development signals an evolution in thinking that is long overdue.

The innovative features of the plan are overshadowed by its adherence to two old, very worn-out shibboleths; namely, that Milwaukee companies have job they are unable to fill and that Milwaukee workers are too unskilled and undisciplined to be good employees.  Each of these is true to some extent but neither is as important as policymakers want to believe.  It only takes one story of a major corporate CEO complaining that he cannot find skilled workers for the policy and funding waters to part.  The blame game then becomes hot and heavy. Elected officials and corporate leaders practically stand in line to take shots at the Milwaukee Public Schools and Milwaukee Area Technical College, never mind the huge numbers of graduates of both institutions who are employed in local government and businesses.

Could both of these institutions do better?  Sure, but so could employers.  In major initiatives like the 30th Street project, employers are frequently asked what they want in workers but they are hardly ever asked about their hiring practices.  There is an assumption that there hiring practices are appropriate and fair.  This leads to the companion assumption that applicants who do not get hired failed to meet employer standards that were appropriate and fair.  This does not describe what is really happening.

Equal opportunity laws forced public employers like police and fire departments to completely revamp their application, testing, and hiring practices to remove bias and facilitate fair employment practices. As a result, diversity in their ranks has increased.   It is time corporate CEO’s who are complaining about worker shortages to look critically at their own hiring practices relative to racial disparities. One place to start is the growing reliance on CCAP (Circuit Court Automation Program) to predict whether a job applicant will be a good employee.  CCAP is an online information system which makes everyone’s legal past available for review. CCAP lists an individual’s traffic tickets, civil judgments, divorce proceedings, as well as felony convictions.  CCAP even lists charges that were later dismissed.  Nowhere has the term ‘too much information’ been more apt that in the case of what employers can learn and use against applicants via CCAP.  Of course employers want to know if an applicant has committed a felony.  Whether a felony conviction should bar someone from employment is another question.  The key thing is that minor things, like speeding or disorderly conduct tickets issues years prior, can be used a reasons not to hire. 

Is it any wonder that our state’s glaring racial disparity in law enforcement – including traffic stops, charging decisions, sentencing, and probation revocation – extends its ugly hands into the employment sphere?  A history of municipal ordinance violations or other legal troubles which has nothing to do with employment history or potential should not be used as a reason to disqualify potential applicants especially when we know how prevalent racial disparities are in law enforcement decision-making.

Our state has convened a Commission on Racial Disparity, funded projects locally to address the persistent imbalance in the application of the law, and tracks its progress in annual reports issued by the Office of Justice Assistance.  Clearly, the existence of racial disparities is not in question. Yet, employers continue to use the consequences of racial disparity, as reflected on CCAP, to keep people out of work.

Milwaukee employers should stop complaining about MPS, MATC, and the pool of prospective workers and take a hard, critical look at their own hiring practices, especially their reliance on CCAP as a primary tool for evaluating prospective workers.  We have let the private sector off the hook for too long.   Now is the time for employers to own up and stand up.

___________

For more information on racial disparities in Wisconsin, go to the Office of Justice Assistance website at http://oja.wi.gov/section.asp?linkid=1344&locid=97


I Could’ve Been a Bat Girl: Notes from Spring Training

Replaying one of my favorite posts from the past because……I’m at spring training.

Of course, how could I have been a bat girl? There ARE no bat girls. Bat people are boys. We all know that. Still. I could pick up bats and keep the ump supplied with balls with the best of them. Because I’ve been to spring training. In fact, I’m at Brewers Spring Training in Phoenix, AZ as we speak. And if there’s a better place to be, I sure don’t know where it is.

I’m not a maniacal baseball fan, nor a student of baseball. However, I am married to an avid fan and attend a lot of games every year – we’re talking 25 or so not counting 3-4 spring training games. Until very recently, watching baseball was a meditative experience for me. But then something clicked – I think it was the day I got the metaphorical significance of Striking Out Looking – and I started to love baseball and baseball players alot.

Spring training is the loveliest thing in the world if you are any kind of a fan at all. First of all, everything about it makes you feel new – new season, new players, new promises. Makes everyone feel like they’re 25. It’s also the most relaxed and mellow place on earth (except for the young guys coming up trying to impress the coaches). There’s a road in Phoenix called Carefree Highway and, in my mind, it runs right to Maryvale where the Brewers Stadium is located. Picture the program vendor who dumps his sack in the 8th inning to stand atop the dugout to lead the crowd in YMCA or the former MPS teacher, now beer vendor, who gives each section a grade on how well they echo his trademark yell.

Most of all, people are happy. The players joke around and tease each other. Prince Fielder has a big grin on his face – something you don’t see once regular season starts. And everyone is kind and chatty and generous. Uncharacteristically, I made a play to catch a promotional T-shirt, missed it, only to have the woman who did catch it give it to me. Dang.

Nothing real profound here. Just Arizona in March with a bunch of young guys playing ball and having fun. Hard to complain. :-)


Creative Repurposing: Lessons from the Prison System

 

The idea of having long-term prison inmates provide care and support for other long-term prison inmates with Alzheimer’s Disease is about as elegant and beautiful an idea as I’ve seen in a long time.  I’m sure its administration isn’t effortless.  There have got to be a million day to day issues that make it challenging, but it seems to be working. As yesterday’s New York Times article, “Life, With Dementia,” suggests, the concept has layered benefits.

The first benefit  layer is the person with Alzheimer’s Disease having a consistent helper who gets to know him, his quirks and worries, and how to calm him and help him negotiate the day.  The second benefit layer is the rediscovery or discovery of life purpose for the inmate who is helping.  If you believe that people can be rehabilitated, that how they were at 20 is not how they could be at 40, then this second benefit is really attractive. Why are we locking up and throwing away the key with no thought about the human potential for service? The third benefit is the changed perspective.  Everyone looks different to the other – prison administrators, helping inmates, inmates with Alzheimer’s Disease.  I think the new prism is respect.

What’s the application for the non-profit world?  We often miss what’s right in front of us – the small solution.  Instead, finding a new need, the first reaction is a new program.  The new program has goals and objectives, performance measures, and job descriptions.  Every step of complexity takes the solution further away from the people having the problem.  And then we wonder why the people with the problem are still hurting. 

 Believe me, I never in a million years thought I would be pointing to something in the prison system as a best practice, but I think this is.  The simple solution – where could we go with that out here in the nonprofit world?

_______________________

Here’s the link to the full NYT story:  http://www.nytimes.com/2012/02/26/health/dealing-with-dementia-among-aging-criminals.html

 


Buried Alive? Clean Your Office Today!

 

No, this isn’t a shot of my desk.  It’s a stock photo of the desk of some hyper-busy, super multi-tasking, too-pressed-to-get-organized person who has since taken up residence in a 19 foot trailer parked deep in the desert outside Yuma, AZ .

Seriously, let’s talk about office hoarding.

With an onslaught of major, major projects over the next three months, I decided recently to clear out my office.  This was a big deal.  I’m a stacker. Well, maybe piler would be a better description. The flat spaces in my office which include a pretty large wraparound desk and a very large table are usually covered with stacks of files, papers, to-do piles.  Because my file cabinets were filled within 24 hours of purchase, there was no room for filing current projects. Hence, more and more stacks – tables, floor, shelves.

Finish a big grant?  Better stack the reference materials somewhere just in case.  Organize a competitive bidding process?  Better keep all the proposals and the score sheets in case questions are asked.  You get the idea.  I can’t throw anything away because I might need it someday.

And just like the nice lady on Hoarders-Buried Alive last night, I cannot stand throwing out something that might make a good project — like stacks and stacks of homeless data/cross-tabs that slice and dice six ways to Sunday (all of which is saved on my hard drive, of course).  http://tlc.howstuffworks.com/tv/hoarding-buried-alive

Some people who live in paper stacks claim that while it looks like a mess, they know exactly where everything is – they can put their hands on a piece of information in seconds.  That’s not me.  I have to rifle through everything to find something.  When I get to the point that I’m spending 50% of my time on a project looking for stuff, I crack.

And then it’s bring in the bags and the boxes, this stuff is leaving.  Now.  Clear out the file cabinet.  Toss the stacks.  Keep the irreplaceable (which is almost nothing these days). Label and file.  Really know where everything is. 

I didn’t have to have the Hoarders team come to my office – but I get  the message.  When your environment interferes with joyful living (or in my case, productive work), it’s time to change it.

Does this hit a nerve for anyone?  What does your office look like?