Another Look at Poll Numbers and Attrition Results
The first thing I need to do today is introduce two tables that show results from our surveys.
This first table shows multiple poll results for the same six months:
Take a look at the first column. If this were a scientific poll then we could expect that September of 2006 would be slightly larger than April of 2006. But once I crunch the numbers, that does not appear to be the case. In fact, the September of 2006 people had recently discovered their log in date, and were (apparently) here in larger numbers than the April 2006 people. Later we’ll see that back in December of 2006 it’s likely we had around 35% of the September 2006 people voting, and only around 25% of the April 2006 people voting. In contrast, in the most recent poll we seem to have around 60% of the April 2006 people voting, and (still) around 35% of the September 2006 people voting.
But before I get to my hypothetical numbers, let’s go back to hard survey numbers. Here are the attrition results taken over the last two plus years for the same six month period:
I made two alterations to those numbers (and I shaded my changes). I changed the 26% because it had been higher than that, and that’s impossible (the attrition rate can’t go up and then back down, people don’t pull out and then come back as a general rule). On that Attrition Survey I gave an April/May/June total, and without going back and finding those data files I have no way of knowing exactly what each month is. I’ve greyed the text on my Hypothetical Chart (below) for this column to show that the figures are more ballpark than exact. The 34% that you see greyed is because that number went down and, as I noted before, that’s not possible. For this reason I’m assuming that the June number for that poll and the one before it are suspect, and those cells show grey text also, instead of black text.
In reality, all of these numbers are ballpark figures, but the grey text is more questionable than the black text.
And now for hypothetical numbers. This is me stretching the limits of what we can do with our figures:
Going back to this post, we can see that there was an average of around 836 dossiers logged in at CCAA per month from January to August of 2006. That is a hard number that came from CCAA, and that is all I can say about it, other than I think it is trustworthy. I used the polls from June ’07 and March ’08 to figure the ratios of how those months would look if we figured January through September all averaged out to 836 dossiers, and came up with the first column in the chart below. That column represents the estimated number of people that the CCAA logged in for those months.
Then, for each poll result section we see the original total dossiers (first column) subtracting out the attrition percentage from the second chart above… followed by a percent showing us the number of respondents on our poll vs the total number of people we estimate are still logged in at that point in time.
Please remember that these percentages are just for these four months – April to September of 2006. We cannot assume that the following months are in the 30 to 35 percent range.
One very interesting thing that popped out at me was that we have more respondents (percentage wise) in the spring than we do in November.
And let me reiterate that the last chart might be close to accurate, but we have no way of knowing whether it is or not. Consider it hypothetical figuring of ballpark figures that come from unscientific surveys. Since I realize the surveys and polls are not scientific, and since I realize what limitations that puts on my numbers, I think I’m okay using the polls and surveys in the ways I primarily use them (short term projections). However, I see people trying to use those numbers to exptrapolate things that can’t be extrapolated from those particular data sets.
That last chart stretches what we can do with our figures to the outer limits. Maybe it’s very close, maybe it’s a long way off. I still think that over all we get about a third of people responding to our polls (when you figure April 2006 through those logged in during 2007, 2008, and 2009) – but it appears we get a whole lot more than that responding once they are close to a referral.
There is actually one more thing we can do with these numbers.
If the CCAA is referring 1500 to 2000 NSN babies per year, then we might expect them to get through April in three to four months, May in three to four months, June in two to three months, July in one to two months, August in two to three months, and September in two to three months. Here is what it might look like – the months listed showing when a month will possibly be complete:
When the State Department issues their 2009 statistics I’ll do that last chart again. For now, it’s a huge hypothetical. We have no way of knowing if the CCAA is going to do those kinds of numbers or not. March had not quite double the amount of LIDs as April, which means logically it should take about five months to get through April. But, we don’t know for sure the attrition percentages for March vs April, either. Also, my Projections show it taking four months to get through April so.. maybe.
In other words, don’t make any sort of plans off of that last chart. When the US State Department issues their numbers I will do that chart again.
December 10th, 2009 at 1:58 pm
RQ–what a wonderful turn of phrase for what you do and we avidly eat up.
“hypothetical figuring of ballpark figures”
Thanks for the information.
December 10th, 2009 at 2:04 pm
I only did this post because I wanted to try to help people see why you can’t compare two polls as if they are part of the same data set. In the process of doing that, we see some interesting figures.. but we don’t know how reliable those figures are.
I think that the Projections that I do each month are more than hypothetical ballparks – they have a history of giving us a good idea of what the next cut off date may be. Polls that are taken in a unscientific way can have value as long as you understand the limitations of the data set. Using the polls for short term projections is fine… but they simply can’t be used for long term projections. And you can’t compare data from two polls side by side. Well, you can, but you are likely going to get faulty results.
December 10th, 2009 at 2:25 pm
RQ – thanks, as always, for all of your analysis.
I think the cell in the last row of the third column should be January 2011?
December 10th, 2009 at 2:27 pm
RQ–there has been an interesting thread in the forum (general discussion I think) about projections that has June 06 referred in 2011 or 2012. I like yours much better and am glad that projections are more than just hypothetical ballparks- but the difference of opinion (or methodology) does make me pause since its been our lot to have the worst case scenarios come to pass. Have you had a chance to look at it?
December 10th, 2009 at 2:39 pm
RQ – you mention ‘short term projections’ above. Would you consider a July 06 LID a short term prediction? I know it is shown in the last chart but wanted to get your take on your feelings for it.
December 10th, 2009 at 2:54 pm
carmel76 – Short Term Projections are dates listed in the first column of my most recent Projections post. In other words – the next four months of referrals, which in this case takes us May 12, 2006 LIDs.
FJM – I have not been in the forum in… too long. It’s been all I’ve been able to manage just keeping things going on the blog. As for discrepancies in what people think may happen in the next batches – I can completely understand why there could be huge differences of opinion. There are so many possible variables right now:
1) Is the CCAA going to keep doing about the same number of days… or are they going to keep referring about the same number of babies? If it is the former then we can make projections based on another Exercise type post, if it is the latter then my Projections are our best bet. One would think it would be based on the number of babies – but once before when we went from a huge month to a small month, the CCAA decreased the number of babies referred in order to keep the days referred at a time steady.
2) Is the CCAA is going to go back to doing about one batch per month, or if they are going to continue stretching out that time frame as well.
Those two variables can make a huge difference in what happens in the coming months. I’m not going to try to make any guesses at this point – I’m just giving possible scenarios while waiting to see what the CCAA does so we know which scenario plays out.
December 10th, 2009 at 3:01 pm
aamum – yes, it should.
December 10th, 2009 at 3:36 pm
Wow! RQ where were you when I was failing prob/stats in college? Thanks for the great explanations.
As a late May LID I am hoping that your projections are on, at this point I think the Jan-Mar referral batches will tell alot about where we stand for 2010.
December 10th, 2009 at 4:25 pm
Thanks RQ!
I’m next……………4-3-06…………..!!!!!!!!!!!!!!!!!
I can’t begin to tell you HOW excited we are!
December 10th, 2009 at 4:39 pm
I am no longer in the “game” but I appreciate seeing your assumptions and methods for calculating projections for “ballpark” referral times.
I think I have been coming here for almost 3 years now…as of 2010. I have a general impression that your projections have been very accurate. However, it seems that in each New Year, for reasons we do not know, the CCAA reduces the number of referrals by some constant rate for NSN. It is so tough to predict but your polling and charts help take out some of the mystery and give us back some control over major life decisions. My new daughter is in my life because of these charts. I took a different path rather than wait 4 years and this resulted in a chain of events that saved lives! Thank you for these calculations RQ. Even estimates are better than absolutely nothing.
December 10th, 2009 at 4:41 pm
Hi RQ. Do you have a rough idea of what percentage of NSN applications are represented by US applicants? I am wondering if it is possible to extrapolate from the State Department data when it comes out to get an idea of the number of NSN referrals worldwide for 2009.
December 10th, 2009 at 4:55 pm
Long_March – yes:
http://chinaadopttalk.com/2008/01/16/comparing-and-analyzing-numbers/
December 10th, 2009 at 5:55 pm
Wow, that’s a lot a work and a lot of info. I’m just super happy to see that Sept 06 is ‘in play’ with projections now! Thank you!
December 10th, 2009 at 6:15 pm
R.Q Won’t the US State Department numbers just list the number of adoptions from China in 2009, and not say if they were SN or NSN? How will you use that number to make projections on the NSN referals?
Will you have any way to know in 2009 how many adoptions per year from China were SN and how many were NSN? Sorry if you already posted this i the past I missed it.
Thanks Glen
http://www.2china4norah.blogspot.com
December 10th, 2009 at 6:53 pm
I really hope you’re right…that would mean I *only* have another 18 or so months to go. After waiting this long, I could do that standing on one foot. ;)
December 10th, 2009 at 8:33 pm
norahs dad – up until now the state dept has not released the difference between NSN and SN.
according to my agency it was 50% last year and is currently at 60%. this jives with what a friend was told directly by c2a2 representatives at an agency meeting.
December 10th, 2009 at 8:35 pm
sorry, i just have to add that it’s so annoying the state dept numbers aren’t out yet. i think they were released last year sometime around 11/18…where are they!?
December 10th, 2009 at 9:25 pm
Wow. Echoing LittleMaplesMom: Is it possible that 9/06ers could begin to see referrals mid-2011? I know – see what happens in 2010, but the possibility is astounding. Thanks, RQ. You’ve given us hope again.
December 11th, 2009 at 1:34 am
i am 9/19/06 and i say ONE YEAR FROM NOW! i never dreamed i’d be this old when we got our second daughter but…what doesn’t kill you makes you stronger!
December 11th, 2009 at 7:52 am
Tearoses 1
I hear you, i didn’t think I would be this old either. But, there is a reason for everything, that is what I tell myself. Our LID is 4/11/06 and I am finally beginning to see the light and possibly get back some of the excitement that has disappeared in the last couple of years.
December 12th, 2009 at 2:34 pm
I agree about the age thing…never in a million years did I expect to be “this old”…so what is “this old”? (48) :-) I have a LID 8-8-06 and so much has happened since I began this journey…loss of parent, loss of job, financially struggling… But wanting to be a mom seems to be so strong that I’m not going to give up. I do think that seeing what happens in April and May will be big determing factors. From what my agency said…we need to just hang in there so we can see the end result. Dropping out just because of the long wait won’t get anyone anything. So why not just hang in there. My only hope is that I can save the money I need. At this time I have no idea on where the money will come from. Maybe I’m just buying time right now. Thanks for all you do RQ.
December 14th, 2009 at 6:25 am
RQ = The last chart, the bottom right date should be January 2011? That makes sense for 2000 referrals per year…..
Thank you for spending so much time for those still waiting. This site is such a blessing.
z
in GZ with baby Sarah on my lap. :-)
December 14th, 2009 at 10:15 am
On November 20, 2009, National Adoption Day, Michele Bond, Deputy Assistant Secretary of State for Consular Affairs held a briefing. The link to the text and video briefing is listed below. I’ve also cut and pasted a question and answer regarding China. In the response, Michele Bond states that the 2009 FY adoptions from China are just over 3,000.
http://www.state.gov/r/pa/prs/ps/2009/nov/132215.htm
QUESTION: Charley Keyes from CNN. You and I have talked in the past. What are some of the remaining problem countries in regard to U.S. adoptions? I know that the United States has voiced concerns in the past about China. Can you just bring us up to date on that?
MS. BOND: All right, yes. To talk about China, which I would not call a problem country as far as adoptions are concerned, but I certainly can talk about that a little bit – in Fiscal Year 2009, just over 3,000 children were adopted from China to the United States. China –
December 14th, 2009 at 10:21 am
For those that don’t know, the state department statistics can be found at
http://adoption.state.gov/news/total_chart.html
FY 04 7044
FY 05 7906
FY 06 6493
FY 07 5453
FY 08 3909
FY 09 3000 *per Michele Bond, State Department