Adding to the debate

In a major speech on Tuesday the Prime Minister will reflect on social exclusion and those trapped in a cycle of poverty, looking at ways to identify and offer early intervention to those families in the most need of help.

In preparation for his lecture, the third in the Our Nation's Future series, Tony Blair considered the following paper - one of those written by a range of experts in the field.

The authors are independent, and all views and opinions are their own. They do not necessarily agree with Government policy, but each makes an interesting contribution to the debate.

• More expert papers

Leon Feinstein

Leon Feinstein is a Reader in the Economics of Education at the Institute of Education, University of London.

Read the paper

Predicting adult life outcomes from earlier signals: identifying those at risk

By Leon Feinstein and Ricardo Sabates

• Read report in full as PDF (opens in new window)

Executive Summary

Overview

This report is about the identification of children at risk of high cost/harm adult outcomes. It summarises findings from the UK Birth Cohort data about the extent to which information about children and their family environments is predictive of later outcomes. The outcomes  are those which tend to be associated with personal harm for them as young people and adults and social cost for those in their environment and wider society.

We find a potentially very high level of predictability in the extent to which we can identify those at risk using early childhood information about family context and child development. In our view these findings are a challenge to which current central and local government should respond with appropriate and measured policy in the interests of social inclusion, personal welfare and the wider economic and social development of the UK.

However, the relationship between childhood risk and high cost or high harm outcomes in adolescence or adulthood is not deterministic, mechanistic or inevitable. There are many steps on the pathway from risk to outcome. There are children at risk who do not experience harmful outcomes and there are children with low apparent or observable risk who do.

Therefore policy responses must allow for flexibility and change. Administrative data should always be augmented by local level, practitioner knowledge and the appropriate interventions should also be selected by local level practitioners who should work closely alongside local communities and agencies to avoid rigid tracking or excessive and unnecessary stigmatisation of vulnerable young people and their families.

Key general findings on forecast accuracy

In the findings reported the method applied is as follows:

1.     We run a statistical regression to predict known adult outcomes on the basis of known childhood risk factors.

2.     Using these results we create a prediction for each individual based on their level of childhood risk.

3.     We select a group of children with the highest level of childhood risk such that this group comprises the same proportion of the sample as does the group that actually experience the adult outcome.

4.     Thus, we have two groups; a group predicted to have the high harm, high cost outcome and a group that does in reality have the outcome. The analysis is based on a comparison of these two groups.

5.     Because not all of the survey participants answer all the questions at all the stages of childhood, we create upper and lower bound estimates of forecast accuracy. The upper bound results are for those sample members for whom we have full information. The lower bound results are based also on cases without full information, giving a larger sample size but lower forecast accuracy.

6.     We have undertaken this analysis for a great many outcomes of adult deprivation in three datasets.

Key findings are as follows:

1.     Using only cases with full information, and using information to age 11 to predict multiple deprivation at age 23, 70.8% of those who go on to experience multiple deprivation at age 23 can be identified at age 11 (the true positive rate). 1.4% of those who do not experience adult deprivation are falsely predicted to do so (the false positive rate).

2.     When cases without full information are also used, the true positive rate is 41.1%, with a false positive rate of 5.2%. The true rates are likely to be between these upper and lower bound estimates, but closer to the upper bound.

3.     For the 1970 cohort, when only observations with full information are used, those with multiple deprivation in terms of 2 or more of 5 key outcomes at age 30 can be predicted in 87.1% of cases using data to age 10, with a false positive rate of 1.1%. When cases without full information are also used, the true positive rate is 43.7%, with a false positive rate of 8.1%.

4.     Using only 5 constructs or pieces of information, assessed up to age 10, we can identify 35% of those who will experience 2 or more of 5 key high cost/harm outcomes at age 30, or, using a slightly different set of 5 constructs, 35% of those who will experience 9 or more from 30 diverse high cost/harm outcomes.

5.     Using only 6 constructs assessed in childhood and also allowing for interactions between these constructs, sub-groups can be created that have probabilities that range between 17% and 77% for multiple deprivation in adulthood.

6.     The probability of experiencing multiple adult deprivation defined in terms of experience of at least 5 of 32 outcomes is 27% for those at moderate to low risk (the 50% at lowest risk), 37% on average and 86% for those defined as at highest risk (the 5% most at risk) based on their childhood family circumstances to age 10.

7.     For higher levels of adult deprivation, this ratio increases. Thus, the probabilities of experiencing 10 or more of the 32 outcomes are 1% for the low risk group, 12% on average and 51% for the high risk group. This highlights the very strong relationship between high childhood risk and multiple adult deprivation. Although none of the 50% at lowest risk experienced 12 or more of these outcomes, 49% of those in the highest risk group did, a very stark difference.

A context for policy responses

In our view it would be irresponsible and socially and economically inefficient to ignore this very high level of capacity to identify early on those at risk of high cost, high harm outcomes. An intelligent system of policy implementation would both respond to these signals of risk and learn from the resulting evidence on implementation to improve its own identification capacity and policy responses.

However, it is also vital to remember that the issue of identification is distinct from that of causation. The fact that we can identify those at risk of high cost/harm outcomes using a small number of key measures does not at all mean that these measures are the causal mechanisms responsible for the outcomes observed or that the measures supply clear indications about what policy initiatives should be employed, when or by whom.

Rather, the measures proxy for a wide ranging set of features of circumstance, development and chance that are the context within which further development takes place. This development is neither deterministic nor inexorable and the degree of predictability does not indicate that the processes that lead to particular outcomes are mechanistic or amenable to obvious, centrally determined interventions.

The particular interventions or supports that will make a difference to the pathways that lead from social address and early development to adolescent and adult outcomes must be applied at a local level and depend on there being skilled and well-resourced local practitioners who are able to make more informed assessments of need and risk than are possible in the national survey data used here.

Just as the Department of Health depends on GPs and other practitioners to make decisions about diagnosis and treatment, so should those concerned with social policy have available to them a system of diagnosis and response that is informed by local level experience and skill. In fact, equivalent if more advanced systems for health care are being developed in the NHS and internationally[1].

The ethics and legality of data matching

A set of very important caveats must in our view also be taken into account in any discussion of appropriate policy responses to the findings in these data, namely issues about the actual data that can or ought to be collected in administrative terms, about the ethics and legality of access to data and about data linking and about the use of the data in the targeting of interventions.

The data used here are not available for use in policy-related exercises and are only for use in scientific research under strict ethical guidelines that ensure anonymity, i.e. that no individual, family or other institution can be identified in any way.

However, information is regularly gathered in schools, doctors' surgeries and elsewhere that in fact might be considerably more predictive for adult outcomes than that collected in the datasets investigated in this research study. Moreover, teachers, social workers and other practitioners routinely form assessments and perceptions of children that can be remarkably accurate about their level of risk. Thus, there is no information barrier to the application of the findings of this study to policy. The barriers are rather in terms of ethical, legal and practical issues that require detailed consideration and debate.

Accuracy is far from total

There will always be false positive and false negatives. This means that early identification cannot be a final or absolute marker of risk and that any system of policy intervention must build in the capability to undertake closer monitoring of risk before intervention is determined and to change assessments of risk in the light of new information, chance events and developmental and contextual shifts. Children change considerably during childhood as do family circumstances and this needs to be allowed for.

It is also to be remembered that the data used in this study were not developed for this purpose and had they been so then it is likely that different measures would have been collected at different ages using a different sample frame. Additionally, there are methodological issues not resolved in this report, in particular the treatment of missing values or missing cases.

Towards an intelligent system for preventative action

We propose in this report a system of policy delivery based on accurate information about individual, family and community context and development. We have shown that with the very rich data available in the UK Birth Cohort Studies it is possible to accurately identify children and families who would benefit from appropriate and effective intervention, were such interventions available. We have also shown that under reasonable assumptions, such interventions are likely to be cost-effective and we have set out a framework for assessing the cost-effectiveness of intervention.

In reality, the type of data and information available to policy-makers will be different to that of the Cohort Studies and so the information is presented as a guide to what might be possible rather than a definitive template. A great many other important longitudinal studies exist around the world and in the UK and each could also provide indications about the most useful topics of measurement. Many other researchers have investigated these issues and would have much to add based on their own analyses of these datasets.

At local level, using administrative data as well as local practitioner judgements such as those of teachers, medics, social workers and others, it would be possible to add to the forecast capability indicated here.

It would not be necessary to collect detailed data on all children. Rather, we propose a system of risk monitoring at which certain levels of risk would trigger greater monitoring and assessment, and ultimately, if judged appropriate, intervention. This is the same process as is followed in relation to medical practice.

The specific measures required

Our findings indicate that measurement of children's own achievements and teacher ratings, particularly from age 7 and beyond can be particularly predictive. Before this age, measurement of developmental health and social and family context will be more important.

The precise measures that are most relevant depend to a certain extent on the outcome of interest. Where these are related to mental health, for example, then earlier measures of mental health will be particularly relevant. For other outcomes such as drug use, worklessness or violence in the home, for example, then different measures may be more predictive. There will be some commonality in the core measures required but it is rarely possible to say that a particular risk factor will lead to a particular outcome. Important childhood risk factors indicate that some degree of multiple adult deprivation is likely and that the individual may struggle with a number of features of adult life. It is often hard to gauge precisely which features of adult deprivation are most likely to be experienced.

It seems likely, therefore, that the most useful framework for developmental measurement and assessment would start from birth with indicators of childhood health and development, together with measurement of family income, education, parenting skill and social ties to the neighbourhood or in terms of wider social and familial networks. As children mature, teacher ratings will become relevant and should be built in. In cases where these forms of measurement indicate high levels of risk, then developmental knowledge of the children's own physical and mental health, behaviour, attitudes and aspirations might be added, together with more detailed information about the family and social context, in order to inform decisions about intervention and support.

The costs and benefits of policy intervention

One reason why it may be appropriate not to respond to these signals of risk would be if there were no appropriate interventions. Other work carried out as part of the Children and Young Persons Review, 2006, strongly indicates that such interventions do exist, provided that they are appropriately funded and targeted on those in need.

In order to clarify some of the costs and benefits of these findings, in Section 6 of this report we demonstrate how great a social saving could be made were appropriate interventions to be carried out, drawing on the previous analysis of identification and targeting. We show that

1. Using prior information on risk of the outcome we can significantly improve the cost effectiveness of intervention by targeting intervention on those most at need.

2. Under reasonable assumptions about the social cost of high cost, high harm outcomes and about intervention effectiveness, our knowledge of individual risk is sufficient to provide a basis for the targeting of cost-effective, early intervention.

3. It is possible to use this information about the identification of those at risk together with the cost-benefit framework set out, to assess the likely cost-effectiveness of any specific proposed intervention to address any specific outcome or sets of outcomes. Use of the framework in this way would provide a common yardstick for intervention choice.

We would recommend use of this system of policy evaluation to assess the likely value of different policy initiatives that may be proposed.

Two issues that need greater reflection and analysis are the questions of stigma and moral hazard. Both present problems for this form of policy system.

1. Stigma. It is very important that early tracking and early intervention do not work so as to reify the problems they are designed to remedy. This would happen if the response to early signals of risk was to create artificial, rigid and exclusive categories. Early prevention will not work if individuals and families simply become categorised as problems. This will exacerbate many of the issues of social exclusion and disengagement that are in part responsible for the high cost, high harm outcomes observed.

However, neither can early intervention be fully effective at remedying the deepest problems of social exclusion if it is only ever voluntary. This has been a long-standing problem for those working in social services who have to make difficult judgements, for example, about when to take a child into care. What we are proposing here is that information be gathered and interpreted more effectively and earlier in the lives of children to make such dramatic and permanent choices less necessary rather than more so.

2. Moral hazard. It must also be a concern that where intervention is of a more positive kind, involving the extra expenditure of resources, then those at risk and others may experience or perceive a benefit from adding to risk rather than reducing it, in order to benefit from the additional resource. Related to this problem is the difficulty of ensuring that resources do indeed go to those most at risk. The inverse care law operates in all areas of social policy, as those with most resources of time, knowledge, social inclusion, income and ambition use those resources to ensure their access to public services, at the cost of those with fewer of such resources. 

Again, these challenges can only be met by a system in which informed and skilled professionals are able to make the important judgements. Moreover, many of the elements of risk that we are proposing be observed would not make rational choices for anyone seeking extra resources and so the moral hazard problem may not be as great as sometimes supposed.

We already have in place a related system that may provide good evidence about what works in such terrain and about the risks and challenges of this type of identification and response. The SEN system is intended to determine the allocation of extra funding and intervention to support the learning of a relatively large proportion of the school population who have particular extra need. In this sense, the system rather mirrors the type of mechanism being proposed here in that there are many forms of SEN and many potential interventions to address these needs.

SEN is a particular risk indicator and the SEN system has its own challenges and difficulties. A broadening of the range of interventions that are triggered by a new set of risk indicators as proposed here would enable the formation of a system that i) identifies, ii) tracks and monitors, and, iii) supports and protects children and families so identified as having levels of developmental risk that are likely to lead to subsequent problems and the further inter-generational and social transmission of the difficulties.