Data

Challenges of Data

VIDEO HERE

Data collection and utilization can be a challenge for any organization. FRCA and our Members Centers are no different. We currently have 30 members in our network and every single one has unique needs when it comes to utilizing their data to the fullest. Each Member Center has differing strategic priorities to leverage and message their data to learn about and serve families. This is in part due to the unique needs of families in each community. Layered on the community needs, are reporting requirements in each of the funding sources our Member Centers receive. All of this variability can make managing our data collection, analysis and evaluation a difficult thing to do. Our systems have to be flexible enough to allow for Member Centers to collect and utilize what is unique about their own communities. In the same way, our data has to be structured enough that we have commonality in identifying not just what services families are accessing, but concretely identifying family progress as families grow alongside our Members themselves.

While we ask our Member Centers to use one database for our entire network, we know that many of our Member Centers are using three, four, or more individual databases to record different data due to funding requirements. Funders have, with genuinely helpful intentions, provided access to services/tools/databases to capture their specific data so that the data is collected consistently across grantees. It has probably been one of the only sustainable ways to collect that information neatly and consistently at scale. However, I have seen the fatigue for Member Center staff when they have to enter the same records in three places. I know enough to see it is not sustainable for the long term. Most Member Center staff work directly with families because they want to work with families, not because they want to enter data in six places afterwards. I’ve seen the same fatigue when data managers are faced with joining four or five data sources to get to what should and could be a simple answer.

While this is a significant challenge we face, there is cause for optimism. At FRCA we are taking on the challenge of data integration from our major collaborating partners to help alleviate that burden of duplicative data entry. We’re investing in and exploring integration projects that could potentially replicate at our Member Centers or even scale to cover the entire network and State. Most encouraging of all, we have the community of data nerds and the culture to make radical solutions work. Our network taps into the skills of economists, teachers, data managers, developers, scientists, and data-lovers who are so much more than a job title can convey. We have a mutual view, that understanding our data means understanding the people we serve and that makes for more effective decision-making. A network this wide and passionate about families, will always find a way to make it work. This includes the challenges of collecting and understanding information to best serve our Colorado communities, Member Centers and families. As we continue to improve our technical supports to Members, there’s plenty of room to grow. I am excited to be a part of this growth and innovation as we wrestle with the challenges of data collection, analysis and evaluation. This journey will take us all places we never expected or dreamed, as individuals, organizations, or as a network.

By Stuart Sims, Data Manager at Family Resource Center Association

FRCA Partners with Hemera and Harvard University’s Center on the Developing Child “Frontiers of Innovation” Portfolio

FRCA has partnered with the Center on the Developing Child at Harvard University’s “Frontiers of Innovation” (FOI) program, and is conducting research on how to grow and sustain a model Family Development program, with a long-term goal of determining how to best serve families on the path to self-reliance by uncovering how families’ needs, community services, and evaluation all align. FRCA is the only statewide Association in the nation working with Harvard’s researchers on how to grow and sustain a model Family Development program, with a long-term goal of improving the lives of children and families.

Frontiers of Innovation is the research and development platform at The Center on Developing Child at Harvard and aims to accelerate the development and adoption of science-based innovations that directly address community-identified unmet challenges of children and families in order to achieve breakthrough impact at scale. Launched in 2011, FOI employs a structured but flexible model that facilitates idea generation, development, implementation, testing and rapid-cycle iteration. This process is grounded in science and supported within a growing community of change agents who are committed to shared learning, cumulative knowledge, and transformative child outcomes at the population level.

“180,000 kids in Colorado are growing up at or below the federal poverty line,” said FRCA Executive Director Mark Kling. “FRCA is honored to work with the experts at Harvard on studying what engages families to utilize support services and how we can create lasting change in our communities by helping at-risk families become self-sufficient. The data we collect through the study will provide us with research on how to make the most impact for the most people.”

The first phase of this partnership focused on reviewing and understanding our data to learn more about what is working and why, as well as what is not working and why not. 13 FRCA Members participated in an orientation meeting that reviewed preliminary findings and invited them to submit applications to be a Cohort Site in the research study.

The second phase is focusing on developing, adopting and testing new, scalable strategies that can achieve significantly better outcomes for young children and families facing adversity, using three primary FOI components: (1) a continuous, bi-directional pipeline of scientific discoveries and hypotheses that will be translated for application in policy and practice: (2) intervention strategies that are designed, tested and refined through the IDEAS framework); and (3) engaging our learning community in shared learning, promoting early adoption of promising strategies, and testing pathways to impact at scale.

The FRCA FOI Rapid-Cycle Learning Cohort (#1) was established through a fair and extensive process in the Spring of 2017 and included three family resource centers across Colorado: Community Partnership Family Resource in Teller, Park, and El Paso counties; La Plata Family Centers Coalition (LPFCC) in rural La Plata and Archuleta counties, and The Pinon Project Family Resource Center in Montezuma and Dolores County. This cohort came to consensus on and then tested a standardized process for recruitment and motivational interviewing-informed goal setting for families newly enrolled in the Family Development Pathway. The cohort also tested the feasibility of adding some of FOI’s recommended measures to deepen our understanding of the Family Development Pathway. In January 2018, Mark Kling and FRCA Program Director, Teri Haymond, introduced FRCA’s Cohort 1 data collection and evaluation process at the Frontiers of Innovation Convening in Boston and discussed the hallmarks of supporting a statewide network of Family Resource Centers.

Lessons learned from FRCA’s FOI Cohort #1 included:

Perceptions that help is not needed, time commitment to engage in services, and lack of correct service matching to family needs were common reasons identified for why families opt out of deeper engagement in service delivery
A quarter of those families that opted into the study dropped out prior to completing, common reasons cited included moving out of the service area and not needing the help;
Motivational Interviewing strategies, a core component of FRCA’s model of family development service delivery, facilitated the development of trusted relationships with families; and
Family progress in self-sufficiency appeared unrelated to parent/caregivers’ level of executive functioning and past adverse childhood experiences, suggesting that services similarly help those with high and low levels of executive functioning and those who have and have not experienced multiple traumas during childhood.

The second cohort of sites, Delta Family Center (Delta) in rural western Colorado serving Delta County and Focus Points Family Resource Center, in metro Denver, will begin recruitment and initial data collection in September 2018. This cohort will test the impact of training additional staff in motivational interviewing (MI) in order to test the impact on family goal achievement of MI being deployed by a broader group of staff. The final conclusions of second cohort are expected in early 2019.

Families are at the center of this work, as articulated in this quote by Teri Haymond: “Families know their situation best and it’s our responsibility to make sure FRCs are equipped to respond to the needs of the people who walk through their doors. One of the most rewarding parts of our partnership with FOI is knowing that our research will help advance the field of family development and help resource centers build relationships with families that will keep them engaged and working toward their goals.”

FRCA has recognized that a broader Colorado FOI Learning Community will be beneficial to supporting this project’s overall goal. To that end, quarterly meetings of both the first and second cohort members will provide a chance to offer input into the project’s design, testing, and reflection phases as well as championing the project outcomes to broader membership.

“Working with Harvard and FOI has pushed FRCA to hold ourselves to the highest standard and ask hard questions about whether or not our work is impactful enough,” said Mark Kling. “This level of research encourages honesty, collaboration and taking risks to achieve great things for families in Colorado and beyond.”

By Teri Haymond, Program Director at Family Resource Center Association

Data is Really About People

August 6, 2018–In 1993, a fledging network of family resource centers emerged out of sheer will and passion to serve our communities where needs persisted and services lagged. The formal association, Family Resource Center Association was established a few years later in 1998. Since then, our network has increasingly used a “data-informed” approach to strengthen services through disciplined practice and decision making.

So here’s the thing. Data in nonprofits is really about people; about our compulsive drive to reveal the truth about our impact; about how we work with people to implement change.

Data-informed practice is people-informed practice.

Several years ago, our network started asking: How do we define our impact? We employed simple data tools to hone in on how many people our network touched with services, directly or indirectly, and into which Colorado communities we reached. We asked questions such as: How do we expand into underserved areas of the state? How can we serve more people in a deeper way? We increased our focus on offering a variety of evidenced-based services in more communities.

As we moved further into data-informed territory, the complexity of tools evolved. Since 2009 our network has used a relational database, Efforts to Outcomes (ETO), to collect participant-level data across the state. (Depending on who you ask, “ETO” might as well be a four-letter word. No one said data collection is easy!) Having a common database helps tell the story of communities across the state that need and value our services.

Every new revelation begs new questions, and we continued to implement more rigorous data practices. Now family development workers use a tool created by social science researchers called the Colorado Family Support Assessment 2.0. It takes into account 14 factors, including: employment and income, housing and food security, physical and mental health, and more. Is it perfect? Of course not. Will it look different five years from now? Maybe. (Gasp!) Does it encourage us to look at what we are doing well, and what we could do better? Absolutely.

Once we had a tool to measure “self-reliance,” the conversation shifted. How would we ensure our work with families helps them achieve progress in areas that are important to them? Family development workers now earn a certification in a technique pioneered by counselors called “Motivational Interviewing.” It turns out that helping people make change in their life requires a similar approach whether it involves giving up smoking or deciding to develop and adhere to a personal budget.

We’ve been offering this approach long enough that we have data to demonstrate that our family development model works. We know that people across the state come to our centers with a vulnerability in one or more of the 14 areas we assess. They also come with strengths and dreams.

Our data, as analyzed by social science researchers that we call “evaluators,” shows that people improve to a more stable place thanks to our strengths-based approach. Hence, here at Mountain Resource Center in Conifer, Colorado, which is also a Mile High United Way Center for Family Opportunity site, we can say that “an external evaluator found that family development participants experienced statistically significant growth in economic self-sufficiency domains.”

We owe it to ourselves, our families, and our supporters to understand the reach and depth of our services. For this, data-informed practice has been, and will continue to be, transformative. At the same time, data-informed practice places additional requirements on our staff whose case loads are full. It challenges our leaders to develop methods for analyzing data even though agencies are stretched thin. We appreciate that our supporters understand the demands of this high-quality, intensive, people-focused approach.

And remember: data isn’t just about numbers. Anecdotes are another type of data we collect. As one of our families recently told us: For most of us, this is enough to keep us striving, out of sheer will and passion, to better understand and meet our community’s needs.

For more information about Mountain Resource Center or the Family Resource Center closest to you, visit our Centers Page.

By Tracy Atlas (Mountain Resource Center)

Challenges of Data

FRCA Partners with Hemera and Harvard University’s Center on the Developing Child “Frontiers of Innovation” Portfolio

Data is Really About People

Newsletter Sign-Up