Evaluating Maryland's Homelessness Solutions Program: A Research Proposal

Background and Rationale for the Evaluation

The proposed evaluation focuses on homelessness. The 2009 Homeless Emergency Assistance and Rapid Transition to Housing (HEARTH) Act defines homelessness as a situation where a family or individual lacks an adequate, regular, and fixed nighttime residence (Substance Abuse and Mental Health Services Administration [SAMHSA], 2023). This definition includes persons living in transitional housing, emergency shelters, abandoned buildings, the streets, and other places not meant for habitation. Individuals who stand to lose their nighttime residence within 14 days, or who are fleeing domestic violence and lack the resources to obtain alternative housing, are also considered homeless under the HEARTH Act of 2009 (SAMHSA, 2023).

Data from the National Alliance to End Homelessness shows that the number of homeless people in the United States has risen by approximately 6 percent since 2017 (SAMHSA, 2023). Contrary to expectations, the COVID-19 pandemic that peaked in 2020 had no significant effect on the U.S. homelessness rate (National Alliance to End Homelessness, 2023). According to a report by the Maryland Interagency Council on Homelessness (2022), this is due to the increase in economic assistance programs and efforts to de-congregate shelters in line with social distancing requirements. Nonetheless, a point-in-time count conducted in 2022 showed that over 582,000 people — approximately 9 in every 5,000 — were homeless across the United States (National Alliance to End Homelessness, 2023). Individual adults made up 72 percent of this population, while 28 percent lived in the streets and other unfit environments with their children (National Alliance to End Homelessness, 2023).

In Maryland, point-in-time count data from 2020 showed that slightly over 6,300 people were homeless statewide, with Baltimore City accounting for the highest rates of homelessness at 35 percent (Maryland Interagency Council on Homelessness, 2022). Generally, the statewide annual count showed that Maryland had approximately 28,288 homeless people in 2020 (Maryland Interagency Council on Homelessness, 2022). Unfortunately, homelessness disproportionately affects special populations, including persons with disability, mentally ill individuals, veterans, unaccompanied in-parenting youth, and victims of domestic violence (Maryland Interagency Council on Homelessness, 2022). Twenty-one percent of homeless people in the state, as per the 2020 point-in-time count, were chronically homeless individuals who had been homeless for over a year and struggled with a physical disability, substance use disorder, or severe mental illness (Maryland Interagency Council on Homelessness, 2022). Homeless veterans made up 7.5 percent of the homeless population, while individuals experiencing domestic violence and unaccompanied homeless youth made up 5 percent and 4 percent of the homeless population, respectively (Maryland Interagency Council on Homelessness, 2022).

Policymakers across the U.S. need to institute ways of addressing homelessness because it increases the risk of mortality and reduces the average life expectancy of affected populations (Maryland Interagency Council on Homelessness, 2022). Studies have shown that homeless people report higher rates of injury-related death and violence than the general population (Maryland Interagency Council on Homelessness, 2022). Data from the Office of the Chief Medical Examiner (as cited in Maryland Interagency Council on Homelessness, 2022) shows that approximately 243 homeless people died in Maryland in 2020. This translates to a death rate of 3,835 per 100,000 people for homeless populations, which is three times the death rate of the general Maryland population (Maryland Interagency Council on Homelessness, 2022). On average, homelessness reduces an individual's life expectancy by 30 years relative to the average person (Maryland Interagency Council on Homelessness, 2022).

Besides mortality, homelessness is associated with a myriad of other adverse health outcomes. The most commonly studied health outcomes among homeless populations include cardiovascular diseases (CVD), HIV and other infectious diseases, communicable diseases, opioid-related overdose, mental health problems, and mortality. A systematic review by Al-Shakarchi et al. (2020) sought to compare the risk of CVD in homeless and non-homeless populations using data from studies drawn from North America and Europe. The findings showed that the risk of CVD and CVD-related mortality among homeless populations was three times greater than that of non-homeless populations (Al-Shakarchi et al., 2020). Sources attribute this to the high prevalence of CVD-related risk factors among homeless populations, including smoking, use of illicit substances, and high cholesterol levels (Al-Shakarchi et al., 2020).

Studies have also found a positive association between homelessness and the risk of HIV, sexually transmitted diseases, and Hepatitis C (Arum et al., 2021). This is because homeless individuals are more likely to engage in risky behaviors such as sharing drug-injection needles and unsafe sexual behavior (Arum et al., 2021). At the same time, homelessness increases the risk of communicable diseases such as influenza, tuberculosis, mosquito-borne illnesses, skin infections, and Hepatitis A because of limited access to vaccination, low health literacy levels, living in congregate settings, and low levels of personal hygiene (Liu et al., 2020). Homeless populations are also associated with higher use of illicit substances, opioid overdose, and opioid-related deaths (Fine et al., 2022). For instance, 65 percent of deaths reported among homeless individuals in 2020 were due to drug overdose and related complications, with opioid overdoses accounting for 88 percent of these deaths (Maryland Interagency Council on Homelessness, 2022). In a study by Fine et al. (2022) examining mortality rates among 60,092 homeless adults between 2003 and 2018, 12 percent of participants died during the study period, with drug overdoses accounting for 25 percent of reported deaths.

Studies have also found homeless individuals to have a higher prevalence of mental health disorders than their non-homeless counterparts (Gutwinski et al., 2021). A systematic review by Gutwinski et al. (2021) found that approximately 12 percent of homeless individuals had either schizophrenia spectrum disorders or major depression, compared to 0.7 percent of the non-homeless population. The study also found a higher risk of mortality among homeless individuals with mental health disorders compared to the general population (Gutwinski et al., 2021). These adverse health outcomes necessitate programs and policies aimed at reducing homelessness, which are often implemented by government agencies or non-governmental organizations.

Many studies have sought to evaluate the effectiveness of interventions that focus on preventing or reducing homelessness. To inform this evaluation, a systematic review of peer-reviewed articles on homelessness prevention interventions was conducted. The search strategy targeted databases such as Cochrane, PubMed, and Science Direct, with inclusion criteria limited to studies published in the last ten years, conducted in the U.S., and evaluating homelessness prevention or intervention programs. The review identified several studies that evaluated programs similar to the HSP (Dwyer et al., 2023; Philips & Sullivan, 2022; Aubry et al., 2020).

Generally, the studies found that homelessness prevention programs and interventions were effective in preventing and reducing homelessness (Dwyer et al., 2023; Philips & Sullivan, 2022; Aubry et al., 2020). However, the studies also found some interventions, or combinations of interventions, to be more effective than others. For instance, Dwyer et al. (2023) evaluated the impact of unconditional cash transfers and coaching support programs on increasing housing stability among homeless individuals. The study found that unconditional cash transfers, coupled with coaching supports and workshops on plan-making, goal-setting, and self-affirmation, improved housing stability within three months better than cash transfer programs alone. Similarly, Philips and Sullivan (2022) evaluated the effectiveness of financial assistance and case management programs in reducing and preventing homelessness among vulnerable populations. The study found that financial assistance coupled with case management increased housing stability and reduced evictions more than monetary assistance programs alone. The primary weakness of these evaluations is that they focus on income intervention programs alone and disregard supportive housing programs, which are also quite common.

In their systematic review, Aubry et al. (2020) address this weakness by studying the impact of both housing support and income assistance interventions targeting homeless individuals. The study found that compared to usual care, housing support programs increase long-term housing stability at a rate of 1.13. At the same time, income assistance coupled with case management improved the number of days a homeless individual is housed by eight days relative to usual care (Aubry et al., 2020).

The primary strength of this body of work is that the studies use comprehensive data collection methods and include rigorous evaluation designs, both of which enhance the credibility of the findings. However, the reviewed studies have a limited scope, focusing on one or two interventions. The proposed evaluation seeks to assess the HSP, which incorporates a range of diverse interventions targeted at homeless people. Moreover, the reviewed studies do not focus specifically on Maryland, and it would be instructive to examine whether findings from studies conducted in other environments can be replicated in the state.

The program to be evaluated is the Homelessness Solutions Program (HSP), which is administered by the Maryland Department of Housing and Community Development (DHCD) as a statewide response to homelessness (HSP Policy Guide, 2022). The program provides funding and technical support to Local Homeless Coalitions (LHCs) and Continuums of Care (COCs), coordinating resources and funding for local service providers serving the homeless population. To qualify for a grant under the HSP, an applicant must be a local government or non-profit entity offering a client-centered approach to homeless services and compliant with anti-discriminatory policies (HSP Policy Guide, 2022). HSP grantees and sub-grantees must also implement formal grievance procedures, demonstrate sound financial management of resources, and implement a sound Homeless Management Information System (HMIS) (HSP Policy Guide, 2022).

The HSP pursues the following goals and objectives as stated in the HSP Policy Guide (2022) and the DHCD Consolidated Plan (2020), respectively:

Goal 1: Reduce the number of families and individuals at risk of homelessness by providing homelessness prevention assistance and shelter diversion.

Objective 1.1: At least 50 percent of unsheltered homeless families and individuals reached through outreach programs will exit to safe emergency shelters by the end of 2024.

Objective 1.2: 100 percent of households and individuals who become homeless as a result of losing their permanent residence will successfully exit to safe shelters by the end of 2024.

Goal 2: Reduce the duration an individual or household remains homeless by supporting them in accessing permanent housing.

Program Description, Goals, and Logic Model

Objective 2.1: At least 80 percent of previously homeless individuals and households held in emergency shelters and interim housing options will transition effectively to permanent housing by the end of 2024.

Objective 2.2: At least 90 percent of households and individuals who become homeless due to losing their permanent residence will successfully exit to alternative permanent housing by the end of 2024.

Goal 3: Reduce the number of individuals or households returning to homelessness by connecting them to community-based resources, natural support networks, and income supports.

Objective 3.1: At least 10 percent of adult HSP beneficiaries will report increased income by the end of 2024 due to the services and support they receive.

Objective 3.2: At least 25 percent of HSP beneficiaries will report improved mental health by the end of 2024.

While there has been a notable drop in homelessness rates in Maryland over the past several years, homelessness continues to disproportionately affect special populations, including veterans, people with physical disabilities, mentally ill individuals, and unaccompanied youth (Maryland Interagency Council on Homelessness, 2020). For instance, the 2020 point-in-time count revealed that veterans made up close to 8 percent of the homeless population, while chronically homeless persons with some disabling condition made up 21 percent of the count (Maryland Interagency Council on Homelessness, 2020). This data underscores the need to evaluate the effectiveness of the HSP in reducing or preventing homelessness among special populations in Maryland. According to Smith and Ory (2014), program evaluation serves two purposes in public health promotion and education: to assess the effectiveness of a program or policy and its interventions, and to identify areas of weakness that could be used to improve overall program quality.

The proposed evaluation would help policymakers determine how effective the HSP is for the target population, including the relevance of its interventions to selected special populations. This would increase efficiency by ensuring program implementers identify what works and commit more resources toward making homelessness non-recurring, brief, and rare for all Marylanders (HSP Policy Guide, 2022). The evaluation would also help improve program quality by indicating the quality of deployed human resources, their areas of strength, and areas requiring capacity-building (Smith & Ory, 2014). Generally, the evaluation contributes positively to the field of public health by providing insights on what interventions may or may not work effectively in a homeless prevention program. The findings will guide program managers running similar programs on best practices, minimizing the risk of errors and wastage.

As Mabhala et al. (2017) point out, "Homelessness is a more complex social and public health phenomenon than the absence of a place to live" (p. 2). Homelessness results from a combination of structural forces — such as lack of access to mental health services, racial disparities, poverty, and lack of affordable housing — and social factors such as poor mental health, family breakdown, domestic violence, and addiction (Mago et al., 2013). Studies have shown that a majority of homeless people have been victims of traumatic experiences and social disadvantage from childhood, including dysfunctional families, neglect, emotional and sexual abuse, physical abuse, lack of psychological support, and disrupted schooling (Mabhala et al., 2017). All these factors serve as possible drivers of homelessness among Marylanders and may not be addressed by providing housing alone. The HSP's effectiveness will depend on how relevant its interventions are to the specific drivers of homelessness in the state. While the HSP may be effective in linking homeless people with interim or permanent homes, failure to effectively address the social context of homelessness may result in high rates of people returning to homelessness upon exiting the program.

Structural factors such as ineffective housing assistance programs, budget cuts affecting federal housing, and rent increases may all increase homelessness rates (Mabhala et al., 2017). These factors are often dependent on the political environment and may affect the operations of the HSP, reducing its overall effectiveness. Thus, it would be prudent to assess the possible effect of these factors during the evaluation to obtain a holistic view of the program's sufficiency and relevance.

Generally, the HSP targets the homeless population in Maryland. However, the program offers diverse support and services targeting different segments of the homeless population. The HSP's support for homeless individuals and households is divided into three areas: Outreach, Emergency Shelter, and Housing Stabilization (HSP Quick Reference Guide, 2022). Outreach services target unsheltered homeless people to link them with critical health services, housing, and emergency shelters (HSP Quick Reference Guide, 2022). Emergency Shelter services target unsheltered individuals living in places unfit for habitation, connecting them with overnight shelters (HSP Quick Reference Guide, 2022). The HSP supports these populations indirectly by providing essential services during their shelter stay and contributing toward shelters' operating costs. Housing Stabilization focuses on preventing people from falling back into homelessness by supporting them in moving into permanent housing (HSP Quick Reference Guide, 2022).

The Housing Stabilization Services segment is divided into three support areas targeting different populations (HSP Quick Reference Guide, 2022). The first area, Rapid Rehousing, targets households and individuals who lack adequate, regular, and fixed nighttime residences. The second area, Homelessness Prevention, targets individuals with annual incomes below 30 percent of the median income who risk becoming homeless but lack the support and resources to prevent it. Finally, the Permanent Housing Case Management segment targets individuals in permanent housing who risk falling homeless for diverse reasons (HSP Quick Reference Guide, 2022).

The eligibility criteria for an individual or household to benefit from HSP services are as follows (HSP Quick Reference Guide, 2022):

(i) The individual or household lives in a place unfit for human habitation or is in a temporary living arrangement such as a motel, transitional housing, or congregate shelter paid for by government programs or charitable organizations.

(ii) The individual or household's annual income is less than 30 percent of the median family income.

(iii) The individual or household lacks sufficient support networks and resources to maintain stable housing without the HSP's intervention.

These criteria define who may or may not benefit from HSP services. To effectively evaluate the impact of the HSP, the same criteria will be used to determine eligibility for inclusion in the evaluation's control group. Annual income, current living arrangement, and ability to maintain stable housing are confounding variables that may affect housing stability among controls, leading to misleading findings. To address this, the evaluators will ensure that control group participants match HSP participants with respect to these confounding variables (Rosales & Atroshi, 2023). Control group participants will only be eligible if their annual incomes are less than 30 percent of the median family income of $98,461 in Maryland, if they are living in the streets or in a temporary living arrangement, and if they lack access to supports and resources needed to obtain stable housing. Controls who do not meet these criteria will be excluded from the evaluation, ensuring that cases and controls are at comparable levels (Rosales & Atroshi, 2023).

The HSP was instituted in 2017 following a decision by Maryland's agencies to develop a central entity to coordinate the activities of Local Homeless Coalitions (LHCs) and Continuums of Care (COCs) across the state (Maryland Interagency Council on Homelessness, 2022). The role of the HSP was to oversee homelessness programs statewide and ensure that they align with federal and national best practices (Maryland Interagency Council on Homelessness, 2022). The formative years from 2017 to 2019 served as the planning phase, characterized by setting goals and objectives, defining the project scope, and developing the DHCD Consolidated Strategic Plan 2020–2024. The program is currently in the implementation phase, guided by the framework, milestones, and targets laid out in the consolidated strategic plan 2020–2024 (DHCD Consolidated Strategic Plan, 2020).

The evaluation requires financial resources of at least $669,300.80, covering staffing costs and benefits, staff training costs, and financing of equipment and supplies. In equipment and supplies, $43,000 will be required to cover emergency clothing or food vouchers, rental assistance vouchers, housing supplies and furniture vouchers, and participant incentives. Staff salaries and benefits amount to $620,760.80, and an additional $5,600 is required to finance training sessions for case managers engaged in the evaluation.

The evaluation will engage two full-time staff and eleven part-time staff. The evaluation will employ a project director and project assistant on a full-time basis. The project director will serve as the primary contact person and will be responsible for recruiting case managers, training consultants, and the program analyst, as well as overseeing the evaluation process. The project assistant will assist in accounting and record-keeping, data entry, and coordinating and scheduling training sessions. The part-time staff will include ten case managers and a program analyst. The case managers will conduct assessments and offer case management services, while the program analyst will collect and analyze data.

The evaluation will additionally require office space. The DHCD agreed to provide 2,200 square feet of office space for the evaluation. The evaluation team will also require technological equipment, including three desktop computers, one laptop for the project director, ten tablets for the case managers, an office printer, and a projector for meetings.

Outputs are the direct, immediate results of implementing a certain activity (Scott, 2023). One of the activities the HSP uses to reduce and prevent homelessness is outreach targeting people living in places unfit for human habitation, with the aim of linking them with emergency shelters. The immediate result of this activity is that homeless individuals and households learn about the HSP and its services and accept to move to emergency shelters. Declines in the proportion of homeless people living in unfit areas will indicate that the outreach program is effective.

Upon moving these individuals out of unfit areas, the HSP carries out a range of other interventions, including assessments to identify income levels, financial literacy, and housing needs; short-term financial assistance; counseling and financial education services; and long-term supportive and housing services. The direct outputs of these interventions are that homeless individuals have increased access to safe and affordable housing as a result of financial supports, a better understanding of how to manage their finances, and improved psychological health as a result of counseling services.

In the short term, one of the outcomes realized by the HSP is increased access to permanent homes as a result of the long-term supports offered. Participants are also able to apply financial literacy skills in making proper budgeting, investment, and financial management decisions, leading to increased income and greater ability to support themselves and their families. Additionally, HSP participants are able to use their improved psychological well-being to improve their social skills and their ability to secure and maintain employment. Within the first six months of HSP participation, individuals would demonstrate a lower risk of falling back into homelessness, thereby realizing the HSP's goal of reducing the number of individuals returning to homelessness.

In the intermediate period, the HSP advocates for effective homelessness prevention interventions among community members and stakeholders to ensure that its partners implement effective interventions. This advocacy minimizes the duration that individuals remain homeless and the risk of beneficiaries returning to homelessness. Further, the HSP increases collaboration among service providers working with homeless individuals to ensure that more people, especially in marginalized areas, benefit from HSP interventions.

In the long term, HSP participants report sustainable housing stability — the ability to maintain stable housing in the absence of the HSP. Ultimately, inequality and poverty rates among participants decline as they become more able to secure and maintain employment or run successful business ventures. The quality of life among HSP participants improves, leading to better health and greater well-being.

The logic model below summarizes the inputs, activities, outputs, and outcomes of the HSP evaluation.

Inputs: Funding allocated to the HSP; staff and volunteers trained to work with homeless participants; partnerships with stakeholders; access to affordable housing; access to financial education resources; evaluation specialist to assist with program planning and data collection.

Evaluation Focus, Design, and Stakeholders

Activities: Outreach and engagement to identify and connect with individuals and families experiencing homelessness; assessment of participants' needs, including housing, income, and financial literacy; delivery of short-term financial assistance (such as rental and utility payments) to prevent or end homelessness; delivery of long-term housing and supportive services to help participants maintain housing stability and improve their financial situation; delivery of financial education and counseling services to help participants build financial literacy and achieve their financial goals.

Outputs: Outreach programs successfully identify and link individuals and households experiencing homelessness to HSP services; available assessments accurately identify homeless individuals' income, housing needs, and financial literacy; short-term financial assistance successfully increases access to safe and affordable housing; long-term housing and supportive services successfully increase access to permanent housing; counseling and financial services effectively improve participants' mental health and financial literacy.

Short-Term Outcomes: Increased number of individuals and families receiving services through HSP; increased housing stability for program participants; improved financial literacy for program participants; increased access to affordable and safe housing; improved psychological well-being for program participants.

Intermediate Outcomes: Reduced rates of homelessness for program participants in Maryland; increased knowledge and understanding of effective homelessness solutions among stakeholders and community members; increased collaboration among housing and service providers to address homelessness in Maryland.

Long-Term Outcomes: Sustainable housing stability and financial well-being for HSP participants; reduced rates of poverty and inequality among HSP participants; improved overall quality of life among HSP participants.

The proposed evaluation seeks to answer the following questions:

(a) Compared to non-participants, are HSP participants more able to secure and maintain stable housing?

(b) How does the financial well-being of HSP participants compare with non-participants at program start, end, and at six-month follow-up?

(c) Is the program accessible and relevant to the diverse needs of homeless people?

(d) Are program participants consistently satisfied with the support and services they receive from the HSP?

The stakeholders in this evaluation can be categorized into primary and secondary stakeholders. Primary stakeholders have a direct stake in the evaluation and include HSP participants, the DHCD board, and staff working directly in the HSP. Secondary stakeholders are indirectly affected by the evaluation findings and include HSP grantees and sub-grantees (LHCs and COCs that work with the HSP), funders, and local communities.

The evaluation findings will be crucial to the DHCD and its staff, grantees and sub-grantees, funders of the HSP, and local communities. To the DHCD and its staff, the findings will provide insights into the overall quality and effectiveness of the HSP's homelessness solutions and possible areas of improvement. The HSP management could use the evaluation findings to analyze program staff performance and identify areas that require capacity enhancement (Harris, 2016). The evaluation findings will indicate how relevant the services and supports offered by grantees and sub-grantees are, and how satisfied beneficiaries are with the services they receive. To the federal government as the primary funder, the findings will indicate whether the program is a worthwhile investment in reducing homelessness in Maryland.

Harris (2016) notes that engaging stakeholders in program evaluation creates buy-in and minimizes resistance in the event that changes are needed to enhance program quality and effectiveness. Thus, stakeholders will be engaged in both developing the evaluation proposal and its implementation. In developing the proposal, program staff, volunteers, grantees, and sub-grantees will help design data collection tools and determine what to include in the surveys and interviews to assess the quality of HSP interventions. During the implementation phase, representatives of COCs, LHCs, and program staff will be engaged in interpreting and drawing conclusions from the findings. Finally, a stakeholder forum will be organized with representatives of all relevant stakeholders to disseminate the project findings, and stakeholders will be expected to share findings with their respective agencies and organizations.

The evaluation uses a case-control evaluation design. According to Tenny et al. (2023), a case-control design provides an effective means to study factors associated with rare outcomes or diseases of interest. It is often used as an alternative to longitudinal approaches when such approaches would require lengthy and large studies that are not feasible to conduct (Tenny et al., 2023). The proposed evaluation seeks to investigate how HSP participation affects housing stability (the outcome of interest) among homeless individuals. A longitudinal design involving repeated data collection from a large number of HSP participants over an extended period would be the most appropriate, but such a design may be extremely costly and complex to implement, especially given the difficulties involved in following up HSP participants long after they exit the program. The case-control design provides a plausible and practical alternative.

The case-control design involves selecting cases (individuals with the outcome of interest) and controls (individuals with similar characteristics but without the outcome of interest), then comparing etiological factors to determine whether exposures occur more commonly among the cases than controls (Tenny et al., 2023). If cases report a more common occurrence of the exposures than controls, one can conclude that an association exists between the exposures and the outcome of interest (Tenny et al., 2023).

Using the case-control design, the evaluator will compare the outcomes of HSP participants (cases) with non-participants (controls) to determine whether the program effectively reduces the risk of homelessness among its beneficiaries. The control group will include homeless individuals who frequent five participating shelters that do not receive HSP grants, yet meet the HSP eligibility criteria. The cases will be obtained from five homeless shelters that receive HSP funding. Comparison will be based on financial literacy and housing stability scores at three points: before joining the program, upon completing the program, and six months after exiting the program.

The case-control design has several strengths. First, it provides an appropriate means to study rare outcomes or conditions; when an outcome is very rare, it may take a long time before a researcher can accrue sufficient cases using other designs (Tenny et al., 2023). Second, the design allows one to study multiple factors or exposures simultaneously, making it less costly and time-consuming than longitudinal or cohort designs (Tenny et al., 2023).

The primary limitation of the case-control design is its susceptibility to recall and selection bias (Tenny et al., 2023). Recall bias is the risk that individuals with the outcome (cases) will recall the presence of exposures better than controls, leading to incorrect conclusions (Tenny et al., 2023). Selection bias occurs when the selected controls are not representative of the population from which the cases developed, increasing the risk of inaccurate findings (Tenny et al., 2023). A third limitation is that case-control designs show correlation but do not establish causation (Tenny et al., 2023). For instance, the design may reveal a correlation between HSP participation and housing stability, but will not establish whether HSP participation is the cause of higher housing stability. To establish causality, other designs such as randomized controlled trials (RCTs) would be required.