A 'tsunami' swamps Archives and Silicon Valley firm serving up 1940 census

Update, 5:40 p.m. ET: The firm at the center of today's census records meltdown says, "We were expecting a flood, but we got a tsunami."

"We had estimates of how much traffic was going to hit the site, and we did performance testing at several levels above that, but we were surprised by the traffic," Joe Godfrey, senior director of product and general manager for Inflection, a Silicon Valley database company."

Inflection was hired by the National Archives and Records Administration, which provided the 1940 census records. Inflection buiilt the search engine to serve up the records, and relied on Amazon Simple Storage Service (Amazon S3) as the cloud service provider. Inflection has been adding more of a pipeline to Amazon all day, adding the ability for more simultaneous connections, but so far searches for census records are running slowly or not running at all for many users.


The company is trying to serve up 3.8 million images of census documents, each with multiple views at different zoom levels, with each file being 10 megabytes or larger.

Godfrey said the situation has improved, and engineers are hoping by the end of today to have the situation squared away.

Earlier:

Embarrassed by a computer system that crumbled under public demand, the National Archives and Records Administration said Monday that it's working to add more servers for the release of 1940 Census records. For more users the wait to see records on family members from the Great Depression era will go on for a while longer.

The Archives had hired Inflection, a Silicon Valley database company, to run the computers, but frustrated users lit up Facebook and Twitter with complaints about images that were said to be "loading" but never arrived.

"Our testing indicated NARA and Inflection could handle the load, but 1.9 mil visitors caused issues we're working to resolve," the Archives said via Twitter. Later it added, "We'll let you know as soon as we have another update - thank you for your patience, we know it's incredibly frustrating."

Even agency officials, during the webcast to kick off the day, couldn't get images to load when they tried to look up their own relatives.

In Springfield, Ohio, Facebook user Val Lough commented on our page: "It's very sweet of them to put all of these records on line. It would be even nicer of them to make the records VISIBLE. None of them will download, I have a browser window opening that's 'loading' the documents and has been for about 20 minutes. You might want to find out what their issues are. It would be faster to mail a public records request to the National Archives." Many others are tweeting about delays.

The National Archives says it is putting more servers online to handle the crush.  At one point, the Archives said, its computers were receiving 100,000 hits per second.

Hey, you've waited 72 years to see these records, so what's another day or two.

Earlier:

A time capsule from 1940 was opened on Monday at 9 a.m. ET, and we invite readers to share what they find. If you use the new records to find information about the loved or lost in your family, please post a note in the comments below or on our Open Channel page on Facebook.

U.S. Census records for individuals from April 1, 1940, protected until now by a 72-year privacy law, are now public for the first time, revealing details about millions of Americans from that day, as the country lingered in a Great Depression, still a year away from entry into war in Europe and the Pacific.

"I'm so excited!" Gary Robert Del Carlo of Martinsburg, W.Va., posted on Facebook. "Maybe for the first time ever, I'll be able to find out something about my father. All I have is my birth certificate with his name, date of birth, state born in, and that he was in the Army stationed in Washington State. His military records burned up in St. Louis in a fire in 1973. They would have told me a lot. Wrote for his birth certificate, and there was no records of his birth. I have done nothing but hit brick walls every which way I turn. I'm praying I find something useful tomorrow, anything."

NPR describes the release as the "Super Bowl for Genealogists." Librarians around the country are ready to provide assistance. At the Family History Library in Salt Lake City, the staff will be serving cake and providing help.

 

When the 120,000 census takers counted 132,164,569 people living in the country on that day, the information collected included the address, whether the house was owned or rented, value of the home or monthly rent, is it considered a farm, names of adults and children, familiy relationships, sex, race, age, place of birth, citizenship, residence five years earlier, education. And for a small subset of people, about 5 percent, they were asked about place of birth of mother and father, language spoken in the home as a child, veteran status, wars served in, Social Security status, occupation, employment status, occupation, number of weeks worked in 1939, income and, for women, whether they had been married more than once, age at first marriage, and number of children ever born.

There is a catch. As the records go online, they can't be searched by name. For a city it's helpful to know an exact address, but often you can work with a neighborhood (near the corner of Canal and Varrick streets in New York City). Your public library may have old city directories or telephone directories from that period, allowing you to look up people by name to find an address. For a rural area, you need to know at least the county and the name of the town or township.

Genealogists, librarians and volunteers will begin the work of indexing the records, which eventually will allow searches by name. Two sites, the commercial Ancestry.com and the Mormon Church's FamilySearch.org, have announced plans to provide indexes to their customers as quickly as possible, with some images going online on Monday. FamilySearch and Ancestry.com started putting images from the Census files online early on Monday, but for now without a name index. 

For now, you must know at least an approximate address to get started. You use that address to find an "enumeration district," which in a big city might be only a few blocks, and would be a larger area in a small town.

Another approach, for those interested in a specific place, is to look at all the records for your block or street. If your area was settled in 1940, who lived there then, and what were their lives like?

Your goal: With that district number, you can look on the Census website at the online copy of the form filled out by the census taker in 1940. In 70 years, it has gone from paper to microfilm to computer.

Here are resources to help you with the search (links open in a new window), though as with most things in life, the key is: Ask a librarian.

 

Discuss this post

Jump to discussion page: 1 2

FINALLY! Now Obama will have some ammunition when he claims this is the most transparent administration in the history of the universe. Praise him for releasing this top secret, life altering data! He really is the closest thing we've had to a saint since Jimmy Carter.

    Reply#34 - Mon Apr 2, 2012 3:46 PM EDT
    HerkDeleted

    Bill I wanted to comment on what numbers mean. In 1940 we were a country of families multiple-people/generation households where today the single house- hold rules. In 1940 most children born were born into families with two married parents where today most children are born into single parent households. In 1940 we ate food that was grown fairly local to our cities not shipped halfway from around the world. These are the big picture numbers. It will show marriage and family,households and housing are quite different from 1940's, children being born into single parent households. That with the increase in population increase use of natural resource's will create new problems socially and culturally. Yes big corporations/banks are laughing all the way to the bank. But my question to you Bill is all this sustainable? We have been a population on the move over the last 72 years. We now populate the west and southwest parts of America. Building cities houses and creating a unsustainable environment in the middle of the desert diverting water and resources where before there were none. We ship resources halfway around the world and we receive resources from halfway around the world. The market system is the most inefficient distribution system in the history of the planet. I don't care how you slice the numbers this is not a sustainable model. My question to you Bill will this continue for another 72 years ? Will someone come out and say these numbers are not sustainable? I don't think so.

      Reply#36 - Mon Apr 2, 2012 4:22 PM EDT

      Another governmental agency that has had 72 years to figure out how to effectively release a public document and they totally blew it! These are the same a-holes who want to run a heathcare system for 300 mil plus Americans? This is one example of why the government should be responsible for as little as possible in our daily lives! PS, I wonder who it is in Silicon Valley and at Inflection that will be responsible for the campaign kick back to the Obama re-elect comittee for this little misadventure? You know there is a tie!

        Reply#37 - Mon Apr 2, 2012 4:32 PM EDT

        Educate yourself before spouting off rubbish.

          #37.1 - Mon Apr 2, 2012 5:32 PM EDT
          Reply

          How many of you realize that the indexing of previous census records (1930, 1920, back to the very first census) has all been done by volunteers? The census bureau has not done it because there is no reason or mandate for them to--what governmental goal would be met by that expense and effort on the part of the government? No, the only reason for indexing is for people to be able to find a record of their ancestors, and so it is genealogy groups and organizations that have volunteered to do it.

          • 2 votes
          Reply#38 - Mon Apr 2, 2012 4:41 PM EDT

          The records I looked at showed the address only no names what good is that?

            Reply#39 - Mon Apr 2, 2012 5:06 PM EDT

            When is this period of covert marshal law going to go public??

            My compromised reputations giggle it away with their careers aligned with other citizens destruction as Poet tries his best to re-trigger the world but the nuts and bolts of the darkest and deepest craft and concealed military power probably dates back to the 60's now infused in everyday life and culture for really no reason at all...

            There might as well have been a similar "investigative" environment on November 19th cause I am not the one giggling it away when @!$%#in PaPa Bush calls the FBI and tells them his best friend killed Kennedy as a similar cottage industry gets set up today with DOD serving as the unacknowledged core and the FBI invokes its general guidelines to suppress all information that intentionally would create a generalized threat to national security with any and every method available even using the Press as a way to create a fanatical cusp out of a quite real and horrible truth....

            Just like back then history did not win -under pretext of not alarming this child like nation as it pointlessly lost 58,000 of its own creating an American experience for others to create wealth and self serving ideology from because we could not carry out our most basic responsibility and now for no reason at all we are forced to live under the most perverted rule man kind has ever seen suppressing this nation's dark side so it simply takes stronger hold through the horror of more "individualized" people...

            Despite how it might compromise your lack of talent quid pro qu it is a fair question -that should not be taboo at all-since it is on official FBI memos...

            Cause they had the same exact cowards back then that would report anything to save their compromised assess that they felt were being held hostage by a deeper intent...

            They would report right over anything at this point...

              Reply#40 - Mon Apr 2, 2012 6:40 PM EDT

              It's funny because I used to watch the history channel and they had the most detailed accounts of battles from centuries ago-down to individual names - and now you may be part of the most massive and controlled operations known to man for the exclusive intent that no one ever is in a position to figure them out...

              ...@!$%#in sad actually because the father you step back the more everyone would realize how pointless these life changing environment are...

              ... Because the FBI Director can still claim there is no lawful proof such environments- with war time intent- even exists exclusively because of how massive and pointless they are in the fist place because between the two you have cowards under the most sadistic form of control...

              How long will this history changing filter be up and running along all forms of communication exclusively so there is no lawful proof it ever happened...

              ...the exact opposite of a democracy and what Constitutional intent ever was because it is far easier than anyone thinks for world governments to suppress any type of knowledge cause somehow my destruction got elevated to the suppression of extraterrestrial life because finding out the DOD used force to suppress it for the last 50 years would bring these criminal interpretations of the Constitution into light possibly having them spill over into other unacknowledged programs like the one responsible for this world wide black out for the rest of modern history for no real reason at all...

              ...and once my belief system fails that the military protects other people's interests and not freedom -some how already made a fad in Poets upside down democracy to hold together the simplest economic theories and gay rights not even realizing they were saying it is ok to have a standard lower than Saudi Arabia across the bottom of this entire nation to protect rich people's interests with punishment worse than North Korea across the entire top - we would ask for a better life closer to the founder's intent...

              ...and the evolution of our alternate history sinks in farther and wider and deeper taking more of our life with it for more shameful and shameful of a reason...

              What a shame...

                Reply#41 - Mon Apr 2, 2012 7:07 PM EDT

                April Fool's to the country!

                  Reply#42 - Mon Apr 2, 2012 8:19 PM EDT

                  This must be the Republican fantasy past, when everybody was stupid and racist.

                    Reply#43 - Mon Apr 2, 2012 8:49 PM EDT

                    For those interested in searching the 1940 census images, this is now possible on Ancestry.com and MyHeritage.com. Use this page: www.myheritage.com/1940census

                      Reply#44 - Mon Apr 2, 2012 9:12 PM EDT

                      What information on a census form is that important to withhold for so long? An address, husband's name, wife, number of children and their names. I can't remember all the questions and information asked, but isn't this stuff "public records"? It seems like people can find out more about you in a county office building - like your names, address, even your real estate taxes - doesn't really make sense.

                        Reply#45 - Mon Apr 2, 2012 10:02 PM EDT

                        What a bunch of whiny complainers. Waaaah hhh Waaah WAAAAAAHHHHH. Try waiting until week two and avoiding the first day rush. I'm sure you'll survive after living your whole lives up til now just fine without viewing the 1940 census. And by the way isn't this free?

                        It's the people like Val Lough in Springfield Ohio who complain when they don't get their instant satisfaction that should be embarrassed acting like children, not the computer company. By the way if it would be faster to mail a public records request and it's so important you see this stuff, then why didn't you ever move your lazy butt and do that before? Too much trouble to get off the couch?

                          Reply#46 - Tue Apr 3, 2012 12:58 AM EDT

                          I would assume that the majority of the complainers are under 40! As "Ed from Philly" stated they are looking for instant satisfaction. I think Ancestry should have advertised the states that would be on-line and that it would be only partial 1940. The should have explained that it would take time to index instead just announcing that 1940 would be available on Apr 2nd. This would have certainly cut down on the number of people trying to access the site. As for blaming everything on the gov't - how many of you will be voting to change it?

                          • 1 vote
                          Reply#47 - Tue Apr 3, 2012 6:37 AM EDT

                          Won't help me much. My grandparents were from a very small rural area, not even a town then, and census workers didn't (couldn't) really account for most people in areas like that. My grandmoter's family we know, but not my grandfather's. His parents both died when he was a child, and he and his siblings were lucky enough to all be taken in by the same woman so they did not have to be seperated. Then the courthouse burned; no more records. I have been trying to get my grandfather's military records for years, but it has been a bitch since I can't provide a copy of his birth certificate. Not that they have one to compare it to...

                            Reply#48 - Tue Apr 3, 2012 12:39 PM EDT

                            Does anyone proofread these articles? It's like a fifth grade essay, including the spelling errors and multiple references added in for "fluff"...

                              Reply#49 - Tue Apr 3, 2012 3:04 PM EDT
                              Comment author avatarColleen Kaytervia Facebook

                              I've read several comments about the 72 years and some people seem surprised by that and the complaints about the lack of attention these records have received in the intervening years. Folks, if you can remember 2 years back, a lot of people and news organizations were up in arms about the invasion of privacy represented by the 2010 census. Hogwash. In 1954 (gotta wonder if the McCarthy trials were the catalyst), the law was enacted to seal the records containing personally identifiable information for 72 years. At the time, that was life expectancy so the intention was that when the records were made public, even most of the babies listed would no longer be alive. And hey, I was enumerator 2 years ago. $250,000 fine and 5 years in prison if at any time in my life I reveal personal information I collected on behalf of the Census Bureau.

                              The hundreds of thousands of hours needed to decipher and index the records of 13 million people? I'm thankful for the volunteers who have undertaken the task. It is truly a labor of love for genealogists and of interest to sociologists (particularly following the migrations of the 1930s). I'm also thankful that our tax dollars were not allocated to this.

                              I found my mom, 3 years old, and her family in Floydada Texas. Thankfully, Floydada is not a big city. I visually scanned about 50 pages before finding them. My next task is finding my father, who was in an Alabama orphanage in 1940.

                              BTW, on Ancestry.com, they're testing an interactive viewer that gives you the option of reversing the colors and adjusting the contrast. Sometimes, white on black is much easier to see/read. And the contrast, if you decrease (not increase) contrast, you can see lighter text that goes white when contrast is increased and limited to black/white.

                              Good luck, everyone. Hope you find who you're looking for!

                                Reply#50 - Tue Apr 3, 2012 3:29 PM EDT
                                Jump to discussion page: 1 2
                                You're in Easy Mode. If you prefer, you can use XHTML Mode instead.
                                As a new user, you may notice a few temporary content restrictions. Click here for more info.