Loading

Evidence and Outcomes

Demonstrating Mission Success

We are working to understand the current literature on poverty, equity of education, and effective management of non-profits, to identify any credible research associated with each of these hypotheses. We refer to this as our “external evidence”. (Please see our attached hypothesis testing rubric.) Our primary work in testing the efficacy of our Theory of Change Logic Model consists of collecting and analyzing data from our own LET’S GO program. We refer to this as our “internal evidence”.

 

Click on the image to open.

The evidence outcomes are linked to the numbered boxes on the Theory of Change Logic Model.

 

Our evidence of mission success is presented in an infographic (above) to demonstrate the current success for change steps 5, 6, 7, 9, 10, and 11.  This infographic and the more detailed data analyses behind it are used to determine our “internal evidence” scores. We intend to continue with both our external and internal testing of each hypothesis associated with each change step in our logic model. We believe in the adage: “What gets measured, gets managed”. We fully expect that our Program Support/Delivery operating model will require continuous improvement in order to achieve the internal test scores of all 15 hypotheses to a sufficiently high level that would lead a thoughtful evaluator and investor to consider the overall LET’S GO Theory of Change Logic Model valid. When this level of confidence in the efficacy of LET’S GO’s outcomes is achieved, we will be in a position to state that LET’S GO is fully capable (with other partners) of “Breaking the cycle of poverty through STEM education and workforce development”. When this occurs, LET’S GO will be in a position to scale our program exponentially.

Simultaneously with our big picture thinking, LET’S GO holds itself accountable to partners, funders and our team as a whole, to deliver on our mission and be able to report how we are accomplishing this mission.  In addition to the efforts made to collect monitoring data (outputs already reported), LET’S GO chose to partner with a third-party evaluator to make sure we are truly doing what we say we are doing.  Policy Studies Associates (PSA) is an evaluation firm used by some of our partners.  As a STEM provider for these partners, some of our results are reported through the PSA Evaluation Report.

The PEAR Assessment Summary

In addition, LET’S GO has partnered with The PEAR Institute: Partnerships in Education and Resilience, at Harvard Medical School and McLean Hospital (PEAR), to measure the impact of out-of-school time (OST) STEM programming on students’ science-related attitudes. Students’ STEM interest, identity and workplace skills (relationships with others, critical thinking and perseverance) were measured using the Common Instrument Suite (CIS) survey which includes a Holistic Student Assessment (HSA).  Program quality was assessed using the Dimensions of Success (DoS) observation tool.  All data collected using the DoS tool, were collected by PEAR-certified observers.  Permission to use the CIS survey and DoS observation tool was obtained from The PEAR Institute at Harvard Medical School and McLean Hospital.

Click this image to read the LET’S GO PEAR Assessment Summary 2016-20218

We know positive informal STEM experiences influence peer and adult relationships (teamwork/collaboration) as well as creative thinking, logic skills, and grit (PEAR). Based on assessments using validated tools from PEAR, our key findings from 2018 (see attached)  show that LET’S GO students consistently out-perform students compared to the national norm sample on several STEM-related areas of measurement (p<.05): Interest, Enjoyment, Identity, Career Interest and Career Knowledge. In addition, on average, over 80% of LET’S GO students report a positive change in 21st century skills (critical thinking, perseverance, relationships with adults and peers) with significantly higher levels of perseverance and critical thinking compared to similar students in PEAR’s national norm sample (p<.05). Results from our 2018-19 school year indicated again that LET’S GO youth, both male and female, reported statistically significant positive change (i.e. mean or median score greater than 3.0) on 7 of the 9 scales on the CIS (p’s <.05).  These scales were: STEM engagement, STEM career interest, STEM identity, critical thinking, perseverance, relationships with adults and relationships with peers. (PEAR, 2019).  No significant change was observed for STEM Career Knowledge or pursuing STEM activities on their own outside of their school environment.  No significant negative changes were found.

Finally, 35.7% of students who participated in LET’S GO’s STEM programming met or exceeded growth expectations on the NWEA Reading Assessment, compared to only 24.4% of students who did not participate, indicating the positive effect of our informal STEM programs on literacy (PSA, 2018). Teachers noted that our “STEM offerings provided the opportunity for students to practice a range of new literacy skills including reading and communication” (PSA, 2018).

High quality STEM programming is key to impacting student outcomes. Using Dimensions of Success (DoS) ratings (PEAR), the quality of our STEM programs was assessed to meet or exceed national benchmarks in the following dimensions: usage of materials, organization, space utilization, STEM content learning, STEM engagement and inquiry. Instructors have observed highly positive changes in their students in STEM confidence in science, technology, engineering, math and social skills as a result of our programs and anecdotal reports suggest that student participation is higher at STEM-related out-of-school events than those of other subject areas. In addition, pre/post assessments from select schools show students in LET’S GO STEM programs improve from 42% to 73% in their STEM content knowledge. Encouraged by the positive results they experience in our STEM programs, many of our students are pursuing other advanced learning opportunities.

At one of our elementary schools, 43% of LET’S GO 5th grade students were accepted into advanced middle schools and 64% of 8th grade students in one of our middle school programs were accepted into a declared STEM-track high school. Teacher/facilitators themselves also reported an increase in their own confidence, ability, and interest in STEM after leading LET’S GO learning experiences (PEAR 2018).

All of these measured and reported outcomes are directly linked to our logic model. This Theory of Change Logic Model holds LET’S GO accountable to deliver on its mission. We are confident in the strides made that reinforce the direction of our organizational strategy to break the cycle of poverty through STEM education and workforce development.

Related Articles:

Theory of Change Logic Model

Evaluation

Let’s GO Boys and Girls PEAR Assessment Summary 2016-2018

LET’S GO Theory of Change Logic Model & Evidence of Mission Success – All documents

Our Clients

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce scelerisque nibh et neque faucibus suscipit. Sed auctor ipsum ut tellus faucibus tincidunt.

Copyright 2019 LET’S GO Boys and Girls, Inc. 501(c)(3) nonprofit EIN: 61-1612453 19 Harness Creek View Court, Annapolis Maryland 21403
Powered by Daasn