Our new model uses advanced machine learning to estimate shelter data more effectively. This allows us to provide deeper insights into trends in animal welfare.

In 2024, SAC released national estimates based on organization and species level medians. These estimates, while thoroughly validated, were limited to high-level trends. To dive deeper in the data, SAC used machine learning to enhance the model.

Machine learning is more powerful because it can a) process and learn from multiple pieces of data, b) find hidden patterns, and c) make predictions that are more accurate for specific situations or groups rather than providing a one-size-fits-all number like the median does.

While our previous estimates captured high-level trends, we wanted to offer the animal welfare community a more precise understanding of what’s happening in different organization types, regions and states. This update supports better decision-making for shelters and organizations nationwide.

The model analyzes anonymized data from shelters and rescues across the country. This includes organization-level data (e.g., intake and outcomes) and species-specific information. Monthly data for intakes and outcomes from more than 6k organizations for the years of 2016-2024 is included in the model to determine intake and outcome totals for all organizations across the country.

Demographic data that was identified to be associated with intakes and outcomes and are therefore included in helping to more accurately predict intakes and outcomes are.

Yes, data security and privacy are priorities for Shelter Animals Count. All data used in our model is anonymized to protect individual organizations.

No, organizations will continue to submit data in the same way. The enhanced model works on the back end to improve analysis and reporting.

We will host webinars and publish blog posts to explain the new model in detail and how it helps improve animal welfare nationwide.

Machine learning is teaching a computer to notice patterns and make estimations based on those patterns.

Some machine-learning models do follow statistical rules such as linear regression models.

Other models do not because they rely instead on how representative the data is that we are using. In real-world applications, we know that it may not be 100% representative. For example, we are most likely oversampling in higher intake organizations based on state-level validation. To assist with this, there are methods to make our data more representative. We are currently down-sampling when training for organizations we have no data for.

Using a more complex machine-learning model allows us to pick up on the more variability and complex interactions

– Data is collected

-Data is cleaned

-Important information is selected to help improve the model

-Multiple model types and iterations are developed and evaluated for accuracy

-Model is tested and validated

-Model is used

-Model is monitored and continues to be improved as new data is incorporated

The model is at the individual organization level and is used to fill in missing months of data based on the organization’s past data and other data from like-organizations (organization type is the same), species, and that have similar demographics in their counties.

Shelter Animals Count’s (SAC) national dataset is a blend of actual and estimated intake and outcome numbers for cats and dogs, sourced from over 6,000 organizations. The model consists of three components:

• + Actual Data: Submitted by organizations directly to SAC.

• + Partial Model: Estimates the number of intakes and outcomes for organizations with incomplete monthly data.

• + No Data Model: Estimates the number of intakes and outcomes for organizations with no data.

To validate the model, SAC used Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE):

• +  MAE: The average difference between estimated and actual values, indicating how far off the model’s estimations are from true values.

• + RMSE: Similar to MAE but gives more weight to larger errors, showing the extent of the model’s prediction errors with greater emphasis on larger mistakes.

Both metrics help determine the accuracy of the estimates. Lower MAE and RMSE values are desired. Here are the values for the Partial and No Data models:

• + Partial Model:

  • – MAE = 14.28

• • – RMSE = 35.98

• + No Data Model:

  • – MAE = 8.3

• • – RMSE = 12.3

Note: Accuracy is highest at the national level and decreases for smaller data points.

For our demographic analysis, we used the SVI index and the American Community Survey (ACS).

• + SVI Data: We considered using data from 2016 and 2018, but determined the effort required wouldn’t justify the minimal gain. The 2022 data wasn’t available, and the 2020 data was selected as it fell within the range of our estimates.
• + American Community Survey Data: The ACS provides both 1-year and 5-year aggregated data. We opted for the 5-year aggregate for 2020, as the documentation advises against using multiple 5-year aggregates for analysis. While the use of multiple 1-year surveys is mentioned as an option, they do not cover all counties and areas due to insufficient sample sizes. The 5-year aggregate includes all these areas.
•  

In conclusion, while obtaining more accurate annual demographic statistics could be beneficial, our efforts are better spent on improving and maintaining representative sampling, especially since our current data is already quite reliable.