I'm among the experts debating this issue. I'm disappointed in Mishel whose work I usually hold in high esteem. I've read Mishel's book and it misleads the reader by ignoring obvious inconsistencies in data on New York City, Florida and Chicago that he relegates to the appendix. In fact, wherever Mishel looks at actual student record data that he deems reliable, he too finds a dropout crisis. This is contrary to his own conclusions based on surveys with admitted problems of years surveyed, sample design, and undercounts. Specifically, Mishel's survey-based estimates of the national rate graduation rates are 15% to 35% higher than the actual record data he argues are accurate in Florida, Chicago and New York City, the places he looked at more directly.
Mishel finds that Florida's four year graduation rate for Blacks is about 55%, and Hispanics about 60%, and these rates Mishel admits are inflated by counting GED recipients as graduates. In Chicago, Mishel finds that Black 19 year old males have a graduation rate of 39% and Hispanic males 51%. In New York City Mishel points to an extended 7 year completion rate for all students of just 60%. The New York City rates he cites are actually about 44% for the 4 year graduation rate according to the State of New York. Mishel ignores the fact that only the 4 year rate meets the requirements for evaluating schools and districts under the No Child Left Behind Act.
These alarmingly low numbers are consistent with the analysis from Chris Swanson and many other researchers, besides Jay Greene, that we have relied upon at Harvard.
Mishel's own numbers indicate a crisis. There is an urgent need to address the crisis facing minority youth. Improving the data collection should be part of these efforts rather than cause for further delay.
Finally, Mishel offers no useful recommendations and would have us wait many years until we have more accurate data before we address the problem. He acknowledges the crisis in urban schools where we find high percentages of minority youth yet appears to want to stay the course, which would continue to put minority students at a great disadvantage.
Daniel J. Losen, Senior Education Law and Policy Associate, Civil Rights Project at Harvard University, co-author of Losing our Future: How Minority Youth are Being Left Behind by the Graduation Rate Crisis
http://www.urban.org/
UploadedPDF...ngOurFuture.pdf
dlosen@law.harvard.edu
I am one of the co-authors of the Economic Policy Institute book disputing the current conventional wisdom that minorities have only a 50-50 chance of graduating from high school with a regular diploma. Here is my quick response to some of the points mentioned.
Sample exclusions in the initial round of NELS:88
About 1.9% of the base year population was considered ineligible for language reasons, mostly inability to complete English-language forms. A significant percentage of these were Asians rather than Hispanics. (270 of the excluded 532 students were Hispanics, 175 were Asians, and the remaining 87 language-excluded eighth-grade students were of another race/ethnicity (neither Hispanic nor Asian). However, most of the people ineligible in the base year were included in later follow-up surveys, so that “cases representing about 74 percent of the base year language exclusions became eligible in either the first or second follow-up” (http://nces.ed.gov/pubs95/95377.pdf). As the manual suggests, “the net effect of these additions to the data is to substantially reduce under-coverage of current and former limited English-proficient students.”
The other undercoverage in the initial round of NELS:88 is of that portion of the special education population that is most severely mentally or physically disabled. However, coverage of this population was improved in the first follow-up by the fact that in the base year ineligibles study, nine of the 23 students excluded because of physical barriers to participation, and 140 of the 322 students who had been excluded because of mental barriers to participation, were reclassified as eligible. Similarly, 49 of the previously ineligible sample members were found to be eligible in the second follow-up followback study of excluded students; of these 49 excluded students, 44 had been previously excluded due to mental disability and 5 for physical limitations (http://nces.ed.gov/pubs95/95377.pdf).
However, there were two other sources of undercoverage of the special education population - 1) exclusion of schools (special purpose schools for the disabled were excluded from the base year sampling frame), and 2) the exclusion of ungraded classrooms in what was by definition a sample of eighth graders.
A recent Department of Education study (http://www.nlts2.org/pdfs/NLTS2_STR6_all.pdf, see article in Education Week, http://www.edweek.org/ew/article...4grad.h24.html)
finds that there has been a significant improvement in school completion and academic performance of youth with disabilities. Comparing the 1987 cohort with the 2003 cohort, the report shows that school completion rate of youth with disabilities increased and the dropout rate decreased by 17 percentage points during this time period. About 70% of the 2003 cohort had completed high school by the time the data were collected. The rate of postsecondary education participation by youth with disabilities more than doubled over this time
(Continuing from above)
So overall, it is unlikely that the graduation rates are overestimated by 5-6 percentage points due to the sample exclusions in the base survey of NELS:88. The initial sample exclusion was about 5.35% of the total population, the inclusion of most of the LEP ineligibles and many of the people excluded due to disabilities would mean that the bias from this undercoverage is at best 1-2 percentage points.
Whether the NELS:88 cohort (Class of 1992) was atypical
The graduation rates obtained from the NELS:88 are similar to graduation rates obtained from other high-quality longitudinal surveys. The NELS shows graduation rates with a regular diploma to be 83% overall, and 74% for both blacks and Hispanics, in the Class of 1992. The NLSY97 – a longitudinal survey of youth conducted by the U.S. Department of Labor (http://www.bls.gov/nls/nlsy97.htm) - shows graduation rates with a high school diploma to be 82% overall, and 75% for blacks and 76% for Hispanics, for people aged 20-22 in 2002.
A simple measure of graduation rate – used by Walt Haney and others - can be calculated by dividing the number of diplomas issued in a year by the number of 8th graders four years earlier. These numbers can be calculated on an annual basis from the U.S. Department of Education published statistics, and show that this 8th grade-to-diploma graduation rate was 77.8% for the Class of 1990, 77.9% for the Class of 1991, 78.4% for the Class of 1992, 78.3% for the Class of 1993 and 77.8% for the Class of 1994. The Class of 1992 had indeed the highest graduation rate among these five cohorts, but the differences are minimal.
A quick response to two of Dr. Losen’s assertions.
Dr. Losen writes in his comments that “wherever Mishel looks at actual student record data that he deems reliable, he too finds a dropout crisis”. While this depends on what one means by the word “crisis”, it should be pointed out that we refer to the NELS:88 survey as the gold standard, and it is indeed a longitudinal tracking of actual students over time (for a period of 12 years). The results from the NELS show that minorities have close to a 75% graduation rate. Similar high-quality longitudinal surveys also indicate that the rates have been steadily rising over the last 30 or 40 years – see Tables 3 and 4 in our book, available at http://www.epi.org/content.cfm/
b...book_grad_rates, which compares NELS:88 to High School & Beyond and NLSY97 to NLSY79 - except possibly for black males during the last decade.
Dr. Losen also argues that “Mishel's survey-based estimates of the national rate graduation rates are 15% to 35% higher than the actual record data he argues are accurate in Florida, Chicago and New York City, the places he looked at more directly.” It is hardly surprising that the national graduation rates for blacks is significantly higher than that for blacks in inner cities like Chicago and New York City. White males in Chicago graduate at a rate of 58% - this does not imply that the national graduation rate for whites is 58%. (Even Greene and Winters’s estimate of white male graduation rate is 74% - see http://www.manhattan-institute.o.../cr_48_t1.htm.)
Our aim in this study has been to create a better understanding of the true challenges we face and the progress we’ve made, and help lead the way to better targeted solutions for continuing to close the remaining gaps. Understanding where we are and how far we’ve come can help identify what has been working in American public education. There are significant problems to be addressed - the minority graduation rates are still low and there are significant gaps in completion between whites and Asians on the one hand and blacks and Hispanics on the other. In some inner cities like Chicago black males have only a 40% chance of completing high school with a regular diploma. However, we believe that unless we know the true picture we are unlikely to correctly address these problems.
Joydeep,
Thanks for responding. About the NELS base-year exclusion report, maybe it's from the incredibly obtuse language in the document, but I read it that the individuals excluded would have been eligible in a follow up, not that they were reincluded.
In terms of improving graduation rates for students with disabilities, do you know the proportion that are standard diplomas? (It's a crazy month for me, and I won't have a chance of looking at the NLTS-2 reports for a while.) Warren, Swanson, et al. generally exclude certificates of completion as well as GEDs.
Roy's Response Doesn't Make it Right:
With all due respect, Dr. Roy avoided my central assertion -- that if you are arguing which is more accurate, a Black graduation rate estimate of 75% (Mishel and Roy) or the estimate of about 50% that the Civil Rights Project at Harvard has cited in published reports, the 50% estimate is better supported by the recent actual student record data on Black graduation rates that Mishel and Roy put forth as reliable.
Dr. Roy's most recent response specifically failed to address the longitudinal Florida student record data that Mishel and Roy's book trumpets as reliable, showing all Blacks in the entire state of Florida as having a 55% "on time" graduation rate (including GEDs).
So it's not just the urban data in their book that supports our 50% national estimate and undermines their 75% assertion. The fact is that the only recent state and district level longitudinal student record data they have used, supports the 50% estimate of "on time" graduation that our Harvard study points to as evidence of a severe problem.
And it baffles me that Dr. Roy would prefer to discuss White males to try and refute my suggestion that the horrific rates for Black males in a large urban district like Chicago, where Mishel and Roy point to a 39% rate , is evidence of a crisis. I'm willing to bet that much higher percentages of Black youth in America attend poor urban schools like Chicago's than do White students.
An "on time" rate for Blacks in New York City may be similarly dismal considering the state officially reports a 44% "on time" rate for all students based on longitudinal individual student record data. Even if we use Mishel and Roy's numbers for all students in New York City graduating at 60%, which is the 7 year rather than the 4 year rate(I'm assuming Mishel and Roy would agree that the Black rates are likely to be lower), that's still evidence of a crisis.
Sherman,
Here's what I could find in the NTLS report referred to in my initial response. I could not find the breakdown of completion by regular diploma or certificate of completion - both are included in the 70% number cited below. But the report says clearly that GEDs are not included. Some important parts of the report are extracted belowe, with page numbers in parentheses. (Note that there are occasional footnotes that have not been included here, so if you want a fuller account, you should check the report itself. For the breakdown of completion into regular diplomas and certificates, you can contact any of the authors.)
Changes in School-Exit Status and Timing (page 2-2)
Findings from NLTS and NLTS2 are consistent with state-reported data showing an increase over time in the graduation rate among youth with disabilities and a corresponding decline in the dropout rate (Exhibit 2-1). The proportion of school leavers who had received a high school diploma or certificate of completion increased from 54% to 70% between 1987 and 2003, and those leaving school without finishing declined from 46% to 30% (p
(Continuing from above)
Although they are referred to here as dropouts, in cohort 1, this group included 6% of youth who were reported to have been suspended or expelled or left school for other reasons without finishing; the dropout rate for cohort 2 includes 1% of such school leavers. The rate of school completion in cohort 2 was the same as that in the general population, 70%, whereas in cohort 1, it was much lower (54% vs. 76%).
Participation in High School Diploma/Certificate Programs (page 4-2)
For the 30% of out-of-school youth in 2003 and the 46% in 1987 who had left high school without finishing, post-high-school education did not necessarily mean postsecondary-level education. Dropping out of secondary school is not an irrevocable decision; young people may still obtain a high school diploma by reentering a regular or alternative secondary school program or by taking an examination to obtain a General Educational Development (GED) credential.
Although cohort 2 youth were much more likely than their cohort 1 peers to have finished high school (see Chapter 2), those who dropped out in 2003 were no more or less likely to have participated in GED or other high school equivalency programs than were dropouts in 1987. Within 2 years of leaving secondary school, approximately one-quarter of dropouts in cohorts 1 and 2 (25% and 22%, respectively) had participated in a program to obtain a high school diploma or certificate.
Postsecondary School Enrollment (page 4-2)
In contrast to the unchanged participation in GED programs, the likelihood of enrollment in postsecondary-level education increased over time. There was a 17-percentage-point increase between 1987 and 2003 in young adults with disabilities continuing their education at the postsecondary level Exhibit 4-1). This marked increase resulted in the overall postsecondary enrollment rate more than doubling, from 15% in cohort 1 to 32% in cohort 2. In 2003, almost one-third of out-of-school youth with disabilities had attended a postsecondary school at some time since leaving high school.
With respect to Dr. Losen’s comments, I would like to make the following points.
1. I disagree with the assertion that “the 50% estimate is better supported by the recent actual student record data on Black graduation rates that Mishel and Roy put forth as reliable”. The only actual student record data on Black graduation rates nationwide comes from the longitudinal surveys like the NELS, HS&B, NLSY79 and NLSY97, the decennial censuses and the CPS surveys – all of which were checked by us. All these point to a much higher black graduation rate than the 50% as believed in the conventional wisdom.
Dr. Losen talks about graduation rates estimated from student longitudinal records in New York City, Florida and Chicago, and argues that examining these will yield the true graduation rates in the nation as a whole. It is true that black graduation rates in some inner cities is much lower than the 70-75% rate that we believe to be the national graduation rate for blacks, and we readily admit that these rates have to go up. But just as the U.S. murder rate is not the same as Chicago murder rate, graduation rates in Chicago are not representative of U.S. as a whole. One cannot make the argument that just because black males in Chicago have a graduation rate of about 40%, the national graduation rate for black males is also 40%. Florida ranks 41st in graduation rates out of 50 states in the nation – as in Greene and Winter’s recent report (http://www.manhattan-institute.org/pdf/
StatesResultTablesFinal.xls). It is obvious that these states and cities are not representative of the overall U.S. population – any extrapolation from these cases alone to overall U.S. graduation rates will be seriously erroneous.
By the way, the 5-year graduation rate for blacks in Florida was about 59.2% in 2002-03, and this includes some GEDs – also people moving into adult education programs and GED programs were treated as transfers and not dropouts.
2. Our book does not trumpet the longitudinal Florida student record data. In fact, it is the other way round. It is the report of the National Governors Association Task Force on state high school graduation data (http://www.nga.org/Files/pdf/0507GRAD.PDF), which included both Dr. Jay Greene and Dr. Chris Swanson as its members, that praised Florida’s data collection – see the section on Learning from Leading States on page 19. We refer to the numbers from Florida only to highlight the differences that exist between graduation rate estimates from tracking of individual student records, and graduation rate estimates compiled by Dr. Greene, Dr. Swanson, and others.
While working on graduation rates based on state and city student longitudinal data records, we became aware that without a true national-level student identifier system, it is difficult to convincingly take account of the ‘leavers’ problem – that is, whether students who leave a school district or state really do enroll somewhere else, or they drop out
(Continuing from above)
Hence we decided to put the discussion in an appendix.
3. We should also clarify that our analysis, like Dr. Greene but unlike Dr. Swanson, mostly focuses on final completion status rather than on-time completion. But it is possible to get an idea of on-time completion from the NELS data, see Table 1 in our book. (While 82% of whites and 93% of Asians graduated on-time with a regular diploma, the numbers for Blacks and Hispanics were 63% and 66%respectively.)
Lastly, we never said that black male graduation rates of 40% in inner cities like Chicago do not constitute a crisis. We believe that the definition of a ‘crisis’ can be different for different persons, and hence have refrained from using the term. But we have readily admitted that these graduation rates need to go up much higher. The issue is one of correctly identifying the underlying trends, so as not to underestimate or overestimate the problem and thereby generate misleading prescriptions.
Two Fundamental Flaws with Mishel and Roy's Study:
This blog debate highlights two of the fundamental problems with Mishel and Roy's analysis: Mishel and Roy don't have great data and they use the wrong standard for evaluating the current health of our high schools, especially with regard to Black and other minority students.
Their strongest assertions are linked to a comparatively small number of student outcomes for the Class of 1992. Their "gold standard," the NELS 88, is a more than 12 year old estimate based on a national sample of about 20,000 students. Those unfamiliar with the survey will be surprised to learn that the actual number of Black students in the sample was approximately 1,456 students. Most, but not all, of these Black survey responses were confirmed by looking at their actual student records. Mishel and Roy say that the fact that most of the surveys were confirmed with actual student records is what makes NELS Golden.
As Dr. Roy points out, in their book and in his comments here, NELS shows that back in 1992 only 63.2% of Blacks students graduated "on time." Researchers who use NELS agree that this number should be adjusted downward somewhat because of bias in the sample of Blacks in particular. This is because Black youth are disproportionately incarcerated, homeless or have disabilities, and would not be accurately represented in the NELS sample which excluded those populations. The reported 63.2% does not reflect any of the adjustments for such bias.
Not only are the sampled data suspect, the extended years graduation rate, the 74% they assert over and over, is the wrong standard. We should agree that the vast majority of high schools are designed (and budget their resources) so that successful students will take four years, from grade 9-12 to earn the credits they need to graduate high school with a real diploma. If we accept that premise, then schools and districts should calculate and report the “on time” graduation rate. As Congress required in No Child Left Behind, the graduation rate is the percentage of students that graduate in the “standard number of years with a diploma.” In regulations the administration clarifies that GEDs and other alternative certificates should not be counted when calculating graduation rates. Other measures of school completion do have important value, but an estimate of a four year rate, which is the “standard number or years,” and based only on diplomas, should be the primary measure we use to evaluate the efficacy of our high schools.
From the start, Mishel and Roy ignore the fact that Chris Swanson’s graduation rate estimate was created purposefully to be used as an “on time” graduation rate to meet the needs of evaluating schools and the requirements of NCLB. Over Swanson’s objections, Mishel and Roy treat his estimate of a four year “on time” rate as if it were a 6 year rate creating an apples to oranges comparison. Further ignoring the fact that Swanson’s numb
Losen Response Continued:
Further ignoring the fact that Swanson’s numbers are for the Class of 2001 and the survey sample they use is for the Class of 1992, they insist Swanson is seriously exaggerating the actual rate. They rarely compare his “on time” rates to the 63.2% or other “on time” rates.
The more important problem is that rates based on outcomes of 5, 6, 7 or more years is not the “standard number of years” for high schools, and is a poor indicator of how well a particular high school or district is performing, or whether a particular education reform is having the desired impact. But Mishel and Roy rarely discuss the “on time” 63.2% figure for the Class of 1992, and prefer the 74% number. If we stick to comparing apples to apples, we would have narrowed the difference between the 4 year "on time" 50% estimate for Blacks that we report based on Swanson’s data, and their 4 year estimate of 63.2%, reducing the difference from 25% to 13% percentage points.
But there are further problems: Mishel and Roy’s survey sample of 1,456 Black students is particularly useless for understanding the recent condition of education in urban school districts, or for poor southern states, where most Black students live. Places similar to Florida, New York and Chicago.
With only 1,465 actual students from across the nation in their 1992 sample, and no state or district results in the NELS to start with, it becomes apparent why Mishel and Roy rely more on "non-gold standard" measures (surveys lacking student record confirmation) to dispute our claim that Black and Latino and Native American students, and especially males from these groups, are experiencing a severe problem in many poor and racially isolated districts all across the nation.
The best way to check the current 2006 value of Mishel and Roy’s national estimates for the Class of 1992, is to see whether their older data accurately represents individual student records today. Unfortunately, that’s impossible to do on the national level. All they can give us are these data from Florida, Chicago, and New York.
In Florida we have actual longitudinal individual student records of tens of thousands of Black students from 2003. Mishel calls these data “much better estimates of graduation rates.” The data show that Blacks had an “on time” graduation rate of 54 % in 2003. I regard this data as upwardly biased, and Mishel admits as much in the footnotes, because Florida counts and reports GED recipients as graduates and also removes from the denominator those students who leave school to attend alternative education programs, when they should be counted against the graduation rate. But, despite the serious concerns that the graduation rate reported in Florida overstates the “on time” graduation rate, it is safe to say that the rate based on individual student records would unlikely be any higher than 54%. The data from Chicago and New York suggest even lower rates for Black studen
Losen Response Part III:
The data from Chicago and New York suggest even lower rates for Black students.
With no racial breakdown, the State of New York reports an “on time” graduation rate of 44% based on tens of thousands of longitudinal student records – Roy and Mishel’s gold vein. It should be noted that students not enrolled for 5 consecutive months when they start 9th grade are removed from the cohort. In other words early dropouts aren’t reflected in the 44% figure. In Chicago, the golden measure yields a 39% graduation rate for Black male19 year olds in 1999 and 30.8 % for Black males in 1998. According to Mishel and Roy, these rates present “an accurate picture of high school completion by entering 9th graders.”
Although these more recent golden rates based on tens of thousands of student fall far below the NELS “on time” estimate based on just 1,456 Black students, it is theoretically possible that we would find very very high rates for Blacks elsewhere to balance out the reliable data from two of America’s largest urban districts and one Southern state. But if that were the case we would have heard about such unusual districts by now. Or perhaps New York, Chicago and Florida are much much worse than any other districts and states as Dr. Roy asserts.
In our series of reports, including Dropouts in California, and Book, DROPOUTS IN AMERICA, The Civil Rights Project has examined studies of individual schools by researchers Balfanz and Letgers at Johns Hopkins, recent longitudinal student record data from Los Angeles, detailed district enrollment data from every state analyzed by Swanson of the Urban Institute, and research from many other scholars. Based on the collection of research, we argue that there is a very serious problem in many schools and districts, especially poor urban schools with high levels of racial isolation. Districts in crisis can be found in every state of the nation.
Not only do a multitude of studies from independent scholars across the country yield results that are consistent with the 50% estimate, but Mishel acknowledges that Swanson’s method yielded nearly identical results as Chicago’s “golden” longitudinal record data, the same sort of data that distinguished the NELS as gold. Moreover, our most recent report on the crisis in California revealed that in Los Angeles Latinos and Blacks combined have a 48% graduation rate again based on “golden” longitudinal individual student identifier data analyzed and reported by Dr. Julie Mendoza. I have no reason to doubt that our 50% estimate would prove to be fairly accurate if we had the golden data we all desire from across the nation. While neither The Civil Rights Project, nor Mishel and Roy, can say with absolute precision just how bad the situation currently is, we assert that we have ample evidence of an urgent problem, one that calls for action, now.
Responses to Dr. Losen’s assertions
Does the NELS have Sampling Problems?
Before getting into details it is important to clarify the issue. The NELS shows a black graduation rate of 75% while the Swanson-Urban Institute methodology that Dr. Losen favors shows a 50% rate. It seems that Dr. Losen is questioning the sampling of the NELS and saying that a failure to properly sample blacks leads to an overstated graduation rate. We have not found any scholar who claims that there is any sizeable sampling problem in the NELS (we have asked Losen in prior private communications to provide us people we can talk to about this, as we take such issues seriously) and no one has ever made an argument that sampling problems could explain a 25 percentage point difference in the black graduation rate. If there were such a problem then the vast literature based on the NELS, at least as regarding blacks, would need to be disregarded.
Now to the details. The NELS:88 sample began with 25,000 students – 24,599 to be precise. Of these, 16,321 were white, 3,011 were black and 3,177 were Hispanics (see A Profile of The American Eighth Grader: NELS:-88 Student Descriptive Summary, http://nces.ed.gov/pubs90/90458.pdf, page 103, Appendix A, A-3). The weighted percentages were 71.4% for whites, 13.2% for blacks and 10.4% for Hispanics.
So the actual number of black students in the sample was over 3,000 rather than 1,456 as Dr. Losen claims. The features of the NELS that make it unique and the best dataset to calculate graduation rates are – a large nationally representative dataset; longitudinal tracking of 8th graders till their high school graduation and beyond; multiple rounds of interview with sample members, resulting in low levels of attrition rates from the sample; and checking of graduation status against actual school transcripts to minimize self-reporting bias.
(Continued below)
Responses to Dr. Losen's assertions (Continued)
Anyone interested in knowing more on the methodology or results from the NELS should check up the NELS:88 webpage at the Department of Education website, (http://nces.ed.gov/surveys/nels88/), which lists all the different publications and products. People should also check the new report by Cliff Adelman at the U.S. Department of Education, The Toolbox Revisited: Paths to Degree Completion From High School Through College, http://www.ed.gov/rschstat/resea...isit/
index.html
Dr. Losen also incorrectly asserts that the NELS sample is not representative because Black youth are disproportionately incarcerated, homeless or have disabilities. Neither of these has any significant effect on the estimates, even the minority ones. First, incarceration is not a problem because we are talking of tracking students beginning in their 8th grade. As the U.S. Department of Justice reports, overall only about 0.2% of all state prisoners were under age 18 – see Bureau of Justice Statistics bulletin for April 2005 (http://www.ojp.usdoj.gov/bjs/pub/pdf/pjim04.pdf). Accounting for incarcerated youth is more important when we talk about educational attainment of people aged 25-29 years old or following people in longitudinal surveys even if they enter prison. However, our results from the 2000 census clearly show that incarceration biases the results only for black males, and even then the effect is not large, certainly not enough to believe that there’s a 50% graduation rate. We either include incarcerated populations or adjust for their presence when we examined household surveys.
A second widely used and respected survey – the National Longitudinal Survey of Youth conducted by the U.S. Department of Labor – actually includes the prison population in its sampling framework and comes to the same conclusions. The NLSY shows that the graduation rate with a high school diploma (excluding GEDs) was 82.2% for people aged 20-22 in 2002 – 85% for whites, 75% for Blacks and 76% for Hispanics (see Table 3 in our book, http://www.epi.org/content.cfm/b...ook_grad_rates)
.
(Continued below)
Responses to Dr. Losen's assertions (Continued)
As I mentioned in my earlier response, the initial round of NELS:88 had under-sampled that portion of the special education population that is most severely mentally or physically disabled. However, coverage of this population was improved in the first follow-up by the fact that in the base year ineligibles study, nine of the 23 students excluded because of physical barriers to participation, and 140 of the 322 students who had been excluded because of mental barriers to participation, were reclassified as eligible. Similarly, 49 of the previously ineligible sample members were found to be eligible in the second follow-up followback study of excluded students; of these 49 excluded students, 44 had been previously excluded due to mental disability and 5 for physical limitations (http://nces.ed.gov/pubs95/95377.pdf).
A recent Department of Education study (http://www.nlts2.org/pdfs/NLTS2_STR6_all.pdf, see article in Education Week, http://www.edweek.org/ew/article...4grad.h24.html)
finds that there has been a significant improvement in school completion and academic performance of youth with disabilities. Comparing the 1987 cohort with the 2003 cohort, the report shows that school completion rate of youth with disabilities increased and the dropout rate decreased by 17 percentage points during this time period. About 70% of the 2003 cohort had completed high school by the time the data were collected. The rate of postsecondary education participation by youth with disabilities more than doubled over this time.
Dr. Losen argues that “researchers who use NELS agree that this number (black on-time graduation rate of 63%) should be adjusted downward somewhat because of bias in the sample of Blacks”. However, we have not seen any researcher using the NELS argue this before – it would be useful to refer to the particular researchers and their studies. As stated above, we have previously asked Dr. Losen for articles we can read or people to talk to who have such views of NELS.
Dr. Losen also argues that because the NELS:88 graduation rates were for the Class of 1992, they are not relevant now and in particular, cannot be compared to recent estimates. However, the Greene and Swanson estimates of graduation rates for the early ‘90s are very similar to their estimates for recent years. For example, for the Class of 2001, Greene’s numbers are as follows - Total 70%, Whites 80%, Blacks 55%, Hispanics 50% (http://www.manhattan-institute.org/pdf/
ewp_08.pdf, Tables 8-11). Swanson’s numbers for the same graduating class are 68% for Total, 75% for Whites, Blacks 50%, Hispanics 53% (http://www.urban.org/UploadedPDF/
410934_WhoGraduates.pdf, page 20). So given that Greene and Swanson estimates for the early 90s were far off from the NELS estimates, they are likely to be significantly off in recent years as well.
(Continued below)
Responses to Dr. Losen's assertions (Continued)
Dr. Losen incorrectly claims that we “ignore the fact that Chris Swanson’s graduation rate estimate was created purposefully to be used as an “on time” graduation rate”. However, it is impossible for Swanson to compute an ‘on-time’ graduation rate because the available data only provides all of the diplomas granted each year- early, on-time, and late. This is something Greene readily acknowledges and Swanson never challenged us on this point at a seminar at the Urban Institute. If Losen wants to believe that the graduation rates are on-time because Swanson has labeled them as such, that is his prerogative. However, we would like to know how that late diplomas are distinguished from others in the calculation.
Dr. Losen wrongly claims that we believe the state and city estimates on graduation estimates to be the “gold vein”. As I mentioned in my earlier response, while working on graduation rates based on state and city student longitudinal data records, we became aware that without a true national-level student identifier system, it is difficult to convincingly take account of the ‘leavers’ problem – that is, whether students who leave a school district or state really do enroll somewhere else, or they drop out. There are often other issues regarding these estimates – mostly related to how different states define different aspects of enrollment and graduation. Hence we decided to put the discussion in an appendix. Note that these problems are not shared by the NELS, which is a national sample.
Our purpose in bringing the state and city estimates to the front was to show that estimates based on student longitudinal records- the same data relied on for enrollment and diploma counts, organized differently and better- yield much higher (ten to fifteen percentage points) graduation rates – both overall and for individual races as in Florida and New York City - than the measures used by Dr. Greene and Dr. Swanson for Florida and New York City. Even in Chicago, where the differences in levels is not very high, Swanson’s CPI and Greene’s measure show quite different trends compared to those based on student longitudinal records, and hence are inappropriate. It is especially bizarre for Losen to claim that the low graduation rates observed in New York City and Florida support his view. In fact, the graduation rates are unacceptably low. However, if Losen relies on Swanson’s and Greene’s computations nationally, and we show that their estimates for these areas are substantially less than better measures, it is hard to understand how that supports Swanson’s and Greene’s national estimates.
Finally, Dr. Losen claims that “a multitude of studies from independent scholars across the country yield results that are consistent with the 50% estimate”. This is incorrect, in that the studies are only done by a handful of researchers – Dr. Jay Greene, Dr. Chris Swanson, Dr. John Warren at the University of
University of Minnesota – and a few others. All these studies rely solely on the administrative data on enrollment and diplomas, and moreover, they introduce population or other adjustments which impart serious bias to their estimates, particularly for minorities. There are estimates from Walt Haney using the same underlying data that show far higher graduation rates. It is not so much that these researchers rely on the CCD enrollment and diploma data: it is the choices they make in using the data that lead them to vastly understate graduation rates, particularly for minorities. It is interesting to note that when Dr. Warren claimed the superiority of his methods to those of others including Greene and Swanson, he did so by virtue of the fact that his estimates came closest to the NELS:88 estimates (see the section titled Validating the ECR in his article at the Education Analysis Policy Archives, http://epaa.asu.edu/epaa/v13n51/...3n51/
v13n51.pdf, page 1
.
A third fundamental flaw of Roy's and Mishel's is that they are myopically concerned with only one thing, data analysis, and would wait until another generation of poor and minority youth are lost before they would consider serious action on a national scale. This paralysis by analysis is deeply disturbing to those of us whose work is dedicated to remedying the crisis faced by poor Black and Latino youth in far too many schools and districts across America.
Typical of our exchange, is this comment of Dr. Roy's, "It is especially bizarre for Losen to claim that the low graduation rates observed in New York City and Florida support his view. In fact, the graduation rates are unacceptably low."
What Dr. Roy fails to realize is that my view, and that of The Civil Rights Project at Harvard University, is that there are unacceptably low graduation rates for Blacks, Latinos, Native Americans and other disadvantaged youth. If the "on time" rate back in 1992 was 62.3% and not 50%, that would still be a crisis. But the fact remains that where we have gained access to tens of thousands of longitudinal student record data (from the current century) those individual state and district “on time” rates have been more consistent with a 50/50 Black and Latino “on time” estimate rather than either the 1992 rate of 62.3% or the 1994 rate of 74%, that latter of which Mishel and Roy would have people believe more accurately describes the condition of education for Blacks in 2006.
Mishel and Roy don’t have data for evaluating schools, districts or states. Nor do they have much to crow about if they focused on “on time” graduation rates. So they revert to apples and oranges comparisons of “on time” enrollment based estimates to extended year survey based estimates. That leaves folks wondering as to why they treat the “on time” information, even the NELS 62.3% rate, as irrelevant to the portrayal of a crisis for Black students in our schools?
Despite Roy’s assertion,The Civil Rights Project does not rely on any one source, or just enrollment data to assert there is a crisis. I’ve already said that an “on time” rate of 62.3% would suffice. In addition, not all sampled data support their insistence that NELS 88’s rates from 1992 or 1994 are the most accurate indicators for 2006. Consider, for example, that according to the 2005 NAEP data, nearly half of all Black and Latino students in grade 8 scored “below basic” in reading. I would not be surprised if most of these non-readers fail to graduate “on time” with a diploma. Keep in mind that below basic is two steps below proficient. This is just more evidence that in 2006, our primary concern about a crisis for Black and Latino students, and the quality of schools and districts they attend, is not exaggerated.
So the most important issue, obscured by Mishel and Roy, is that our public schools, K-12 are not providing adequate educational opportunities for these minority students, while subjecting
Losen Part II
So the most important issue, obscured by Mishel and Roy, is that our public schools, K-12 are not providing adequate educational opportunities for these minority students, while subjecting them to unjust high stakes exit exams and encouraging students to dropout rather than stay in school. The “on time” rate is more important for evaluating our schools and districts, especially in the face of these problems and the disproportionate burden they impose on poor and minority youth.
We know the NELS has great value for some purposes, but ultimately it is also a survey with sampling biases and other shortcomings. In our book Dropouts in America: Confronting the Graduation Rate Crisis, Phillip Kaufman writes, “Since NELS is a sample survey, it is subject to the same potential for bias due to non-response and undercoverage bias that the CPS has.” (p. 119-120) He also points out that that the NELS was designed, “to provide national estimates of dropout and completion rates and, except for very large states, cannot provide statistically reliable state estimates or any school district estimates.”
We join Mishel and Roy to the extent that we aren't happy with the current lack of clear and transparent data, and we both call for much better tracking of individual students to get a much better understanding. Apparently they don’t believe the "on time" graduation rate is the most important for evaluating schools and education reform policies. I'm disappointed that Mishel and Roy give no significant time whatsoever to the “on time” rate of 62.3% when they boldly insist there is no crisis. Instead they submerge that piece of information and repeatedly highlight the least useful estimates for describing the state of education for Black students, that's the 6 year, 74% rate based on a small national sample of students where fewer than 1,456 of the surveyed responders actually had their student records confirmed. I strongly urge Mishel and Roy to consider the reaction of the White middle class if they learned their "on time" graduation rate was 62.3%.
On another more technical note: Roy claims the number in NELS is actually 3,000. But after attrition, according to a reputable NELS researcher, for Blacks, the number in the sample that started at about 3,000, was reduced for a variety of reasons to1,456 by 1992 – at least as far as the “golden” group of students for whom most had responses that were checked against actual student records. So while Roy insists I made an error, and skirts over the attrition in his blog response, his book shows how the original 25,000 in the sample shrunk to about 20,000 or less, as detailed on page 73 of Roy’s appendix. If Roy insists the gold standard is based on 3,000, I’d accept those numbers if he takes an oath that there was no attrition in the Black sample and that nearly all of the 3,000 original Black students not only remained in the sample, but had their transcripts checked to confirm the
Losen Part III
If Roy insists the gold standard is based on 3,000, I’d accept those numbers if he takes an oath that there was no attrition in the Black sample and that nearly all of the 3,000 original Black students not only remained in the sample, but had their transcripts checked to confirm their survey responses in 1992 or 1994. What I’d really like to know from Roy is just how many of the 3,000 original Blacks had their survey responses checked against their school records in 1992. That number, which would be the only “golden” data they have for Blacks, is likely even less than 1,456.
Instead of acknowledging the weaknesses of the data they use, Mishel and Roy repeatedly prefer to mislead the public by posting op-eds that call the crisis “exaggerated.” To their credit, you can find the data that undermines their assertions in their own book, but rather than insisting that honorable researchers like Swanson and Warren are exaggerating, they should give equal time to the fact that that "on time" graduation rates are unacceptably low for Blacks and Latinos according to NELS, and by any method, and that the survey data they use offer little current or specific information on the current health of increasingly racially isolated schools and districts serving predominantly poor Black and Latino youth.
In summary, Mishel and Roy don't have great data, and give far too little attention to “on time” graduation rates. Ultimately, they miss the point. For Black, Latino and other disadvantaged youth, there is more than enough information depicting low graduation rates or corroborating such poor outcomes, to call the situation a crisis and urgently demand an improvement in educational opportunities.
Correction to Above:
Please change all references to the "on time" graduation rate from NELS written as 62.3% to 63.2%.
Larry Mishel Responds
This is a tag team. I am taking the response to Dan Losen over from Joydeep. This is difficult to do for several reasons.
Personal Attacks
First, Losen’s tone is way off the reservation and his personal attack is shameful. Consider how he starts his last contribution, saying Joydeep and I:
“….are myopically concerned with only one thing, data analysis, and would wait until another generation of poor and minority youth are lost before they would consider serious action on a national scale. This paralysis by analysis is deeply disturbing to those of us whose work is dedicated to remedying the crisis faced by poor Black and Latino youth in far too many schools and districts across America.”
I am not sure what his basis is for saying we do not favor any serious action on a national scale, that we see no problems or that we do not care about disadvantaged students? We certainly have stated that there are too many dropouts, especially for low-income and minority students. We highlight the existing racial and ethnic gaps in graduation and show the black-white gap is even larger than what the regular CPS data show (the Hispanic-white gap, however, is smaller than previously thought, though still large). Is it because we do not scream about an undifferentiated across-the-board high school dropout ‘crisis’? As for policy implications, you can read below what we think they are. And, has Dan Losen cornered the market on concern for minority youth? Puhhhleeeze. Dan is only embarrassing the worthy organization, the Harvard Civil Rights Project, with which he’s associated. At EPI, which I lead, we speak truth to power every day of the week. We certainly do not accept any lecture by Losen on our commitments or motives. Because of Losen’s tone and personal attacks this will be our last attempt to intellectually engage him—there will be no more postings from us.
A Useful Engagement
A second difficulty is that Losen does not practice empirical social science (as far as we can tell, he does not do any data analysis) so he does not discuss measurement issues in ways that allow useful engagement. How could that be done? For starters he could identify his critique of our evidence by stating what he thinks is wrong with the data we rely on or how we analyzed the data. He could then identify the size of the error or bias result from any flawed data or methods. He might even acknowledge weaknesses in the methods he endorses and try to reconcile estimates of graduation rates. Losen, rather than engage in this manner, dismisses empirical findings and whole datasets based on the simple assertion of a bias. That is, Losen blasts out critiques without any assessment of whether they matter worth a darn. They do not.
Joydeep tried to jumpstart a useful discussion at the start of his last contribution:
‘Before getting into details it is important to clarify the issue. The NELS shows a black graduation rate of 75% while the Swanson-Urban Institute methodology that Dan Losen favors shows a 50% rate. It seems that Losen is questioning the sampling of the NELS and saying that a failure to properly sample blacks leads to an overstated graduation rate. We have not found any scholar who claims that there is any sizeable sampling problem in the NELS (we have asked Losen in prior private communications to provide us people we can talk to about this, as we take such issues seriously) and no one has ever made an argument that sampling problems could explain a 25 percentage point difference in the black graduation rate. If there were such a problem then the vast literature based on the NELS, at least as regarding blacks, would need to be disregarded.”
As far as I can tell, Losen has never made any claims that his alleged problems with the NELS can explain the difference between a roughly 75% graduation rate in NELS and the 50% rate he seems to be comfortable with. Nor has he identified any researcher or research paper that has made a serious (actually, any) critique of NELS. It is curious that Losen cites Kaufman saying the NELS can’t produce state or local estimates. First, Kaufman actually identifies the NELS as the best source of data. Second, the point Kaufman is cited for- NELS doesn’t produce local estimates- is not an answer to why Losen’s preferred method for analyzing local data (the Swanson measure) yields a 25 percentage point lower graduation rate for blacks at the national level than that of the NELS—if Swanson’s methods are so far off at the national level they can’t possibly be worth anything at the local level.
What we have offered
We have compiled or analyzed all of the data we could find, assessed it, and tried to correct biases that we could identify (accounting for high and rising incarceration of black men; including the military and institutional populations into the analysis; develop estimates of GED receipt from ACE data; examine sampling coverage issues and more). We have presented the best estimates from household survey data, including the decennial census, and from national longitudinal data such as the NELS and the NLSY. We find that all of these other data show graduation rates for minorities that are far higher than those found by Greene and Swanson—the conventional wisdom.
We have specifically examined the methods used by Greene and Swanson in their analysis of enrollment and diploma data and have shown how specific choices they make in using the CCD lead them to seriously understate graduation rates, especially for minorities (in our report,
http://www.epi.org/content.cfm/
b...book_grad_rates , and in a Q&A, http://www.epinet.org/books/
reth...ation_rates.pdf ).
Swanson’s measure is the one Losen likes the best. We have shown how Swanson’s use of ninth grade enrollment as his estimate of those entering ninth grade is wrong: around 20% of minorities in ninth grade have been held back and are not ‘entering’ ninth graders. The consequence of using ninth grade enrollment instead of eighth grade enrollment (to avoid what is called the ‘ninth grade bulge’) is to artificially lower minority graduation rates by 12-13 percentage points. We have heard no rebuttals of this critique from Losen or anyone else. We think this is because there is no reasonable response.
So What?
Does it really matter whether the overall graduation rate is 83%, as we estimate, or the now conventional number of 67-70%, as Greene and Swanson (and the National Governors Association) claim? After all, estimates of lower dropout rates still represent a greater than desired population of dropouts.
Our findings do not negate that there are definitely places and populations where dropping out is far too frequent – in some, horrifyingly frequent. There are definitely dropout problems that require comprehensive action so as to improve students’ life chances and to address class and race/ethnic gaps in graduation. But there is also an important value in getting the facts correct and especially in recognizing improvements over time. Touting a 50% graduation rate for minorities is not only factually incorrect but can also too easily encourage mischaracterizing the black student population as too hard to reach or as being disinterested in education. Identifying that three of every four black youths get a diploma and another 13% receive a GED is both factually correct and appropriately acknowledges the striving and persistence of black students.
Artificially low graduation rates also lend themselves to supporting a misdirected across-the-board indictment of schools. Inaccurate characterization of success in high schools can lead to misguided or wrongfully-targeted reform efforts. It can lead to efforts similar to sending homeland security funds to Montana and Idaho rather than to New York and Washington. Recognition that there has been progress in improving graduation rates might lead to an examination of what some schools must be doing right, practices that might be supported and extended to other schools. A misguided across-the-board indictment of schools might also lead to equally misguided radical reforms that could be harmful.
In truth, the dropout problem is concentrated in about 20% of our high schools. If we examine graduation rates by socio-economic status, we see that there is only a 3% dropout rate for the upper three-fifths of students, by socio-economic status. (This calculation includes GED recipients as high school completers). Yet, the bottom fifth had 27% who failed to complete high school in any way, and only 62% who obtained a regular diploma. The next higher fifth had 13% who failed to complete high school. So, there are definitely populations that are not successfully completing high school and getting diplomas, which is bad for those children and bad for the country.
Interestingly, among students in the lowest socio-economic fifth, black students have the highest probability of completing high school, greater than that of low-income whites and Hispanics. This is another reminder of how intertwined race and class are in our society.
To the extent we have a dropout crisis; it is primarily a crisis of youths at the bottom of the socio-economic scale, regardless of race (although race is clearly the key factor
To the extent we have a dropout crisis; it is primarily a crisis of youths at the bottom of the socio-economic scale, regardless of race (although race is clearly the key factor in allocating students to the bottom fifth and plays a far-too important role in our society). To address their problems we need comprehensive efforts to improve these children’s schools, alternative programs and the building up of second-chance systems. But we also have to think about the lives of these students outside of school and the disadvantages they faced even before they ever got to school (there are huge education disparities by income and race when students start kindergarten!). This means fighting poverty through better jobs and wages, providing early childhood development programs, creating stable housing, providing health care and fighting crime. That is certainly a national program we think is worth fighting for every day.
Commenting by HaloScan
