Here’s a question for EI leadership: How do you know if your program is achieving positive outcomes for children and families?
This question is an intricate one for local system managers and other leaders. The answer lies at the heart of what we do.
Child Outcomes – How Do You Know?
On the surface, you could say, sure we are…kids are making progress, aren’t they? Consider this – in research, there is a confound called “maturation.” This confound describes the natural developmental progress that children make over time. How do you know whether or not the supports your staff provide positively affect development, or if the child’s progress was just due to maturation? It’s a hard question. We can never really know, 100% for sure, that EI “caused” a child to make faster or better progress, because EI services are just one variable in a child’s life. We do know, however, that participation in EI is associated with positive developmental outcomes for children.
One of the “big picture” goals we have for infants and toddlers with developmental delays is to see the gap between their level of delay and their chronological age closing. Another goal is to see children achieving the outcomes on the IFSP. A third goal involves helping families improve their children’s participation in activities that are important to them. Of course, we also monitor the three national OSEP child outcomes, which are designed to provide a global indicator of child progress as a result of participating in EI. Much of this is monitored by EI leaders via record reviews, where indirect information is gathered to help us evaluate child outcomes. While important, focusing primarily on data in a child’s record might not allow you to really dig in to find out about the impact of your program’s services of child and family quality of life. At its core, measuring outcomes is all about quality – quality of life for children and families and quality of services provided by your program.
Family Outcomes – How Do You Know?
Measuring positive outcomes is really dependent on the family. Again, we have the OSEP family outcomes, which are designed to help us have specific and consistent ways to measure the impact of EI on families. These are helpful (and required), but I challenge you to think more individualized. How do you know that the services your program provides improve outcomes for each enrolled family? How do you monitor and measure this? Perhaps you can assume that, if you don’t have too many families calling you with concerns, that maybe your program is successful. You can ask your providers about their experiences with families. You can read contact notes and IFSPs. Again, these are indirect ways to monitor outcomes and I doubt that they get at the heart of the impact of EI on families. What else can you do to dig deeper, to really find out about your program’s impact on the quality of family life?
5 Strategies for Answering the Big Question
Consider these strategies and think about what you could do in the new year to dig deeper into the outcomes children and family experience as a result of participating in your program.
Make time for direct observations and conversations – In general, I think many leaders don’t have enough direct contact with families. We get wrapped up in monitoring program operations, putting out fires, and day-to-day supervision. There’s nothing like seeing EI in action and talking directly to families about the impact of services on their daily lives. Plan for regular observations of visits – even if you only do 1-2 a month. Make calls to families to find out about their experiences with your program. You can then share what you learn with the staff to celebrate the positives and grow from constructive feedback.
Use tools to monitor quality…don’t just “eyeball it” – If you are experienced, you may be able to just “eyeball” a visit and feel confident that it’s a good one. Take it a step further and use a checklist to specifically capture the strengths and limitations you see. This will help you debrief with your staff and help your staff have something concrete to refer to. Having a concrete measure of mastery is critical to supporting adult learning and can easily be achieved by accessing great resources like the new ECTA performance checklists, which are designed to cover lots of recommended practices.
Monitor discharges – Keep an eye on reasons for discharge and patterns across entry and exit ratings for the OSEP child indicators. It’s a great sign when you have more families requesting earlier discharge because they feel confident and competent in addressing their child’s development. You can find this out from contact notes, an exit survey sent to each family at discharge, and spontaneous calls to families who’ve left the program.
Stay current with regular child & family updates – Conduct regular 1:1 and group supervision to hear how things are going in the field. Keep an open-door policy and follow-up with staff about successes and challenges they share. Keep your ear to the ground to get a measure of how children and families are doing.
Stay current on evidence-based practices – In order to know if your staff are achieving positive outcomes, you must know what effective interactions with families look like. A good leader does more than manage program operations…he or she leads the program with an eye on the big picture, which is improving outcomes and quality of life for children and families.
How would you answer the big question?
How do you know if your program is achieving positive outcomes for children and families?
Share your thoughts and strategies in the comments below!