Accelerate your Search Beyond Google to find US Government Information?
Do you know that Google and other search engines can’t find much of the information in the databases hidden behind HTML Forms such as search boxes or logins!
Google offers this explanation on their Help pages:
- Making sure your content is text-based and readable helps make it more accessible to humans and Googlebot.
Much of the information and data our US Government publishes each week is not indexed by Google and other search engines because it is accessed using HTML Forms To be fair, some US Government websites use either the Google or Bing custom search service and making that information available in their search engine indexes.
The vast majority of all US government sites do NOT use either Google or Bing search services and most search engines do NOT INDEX the majority of this US Government information.
Using just Google or Bing search services you may miss important information critical to your project!
Let me illustrate this Google and Google Scholar limitation with a particular example comparing results for 10 of our US Government Top-Level Federated Sites including Google and Google Scholar limited to just domains using .gov or .mil.
- fuel – carbon – fiber – composite – tank
Click each database name to see the current results screen for each search – Please note, the number of results you see when you click one of these links may be different from the results we listed from the time we composed this blog. THESE SITES ARE ALWAYS INDEXING NEW INFORMATION, so their results change over time:
- BusinesUSA.gov – returned 2,219 documents.
- Congress.gov – returned 74 documents.
- DATA.GOV – returned 11 datasets.
- Energy.gov – returned 32,380 documents.
- Google Scholar for US Gov – returned 981 documents.
- Google for US Gov – returned 87 documents.
- NCBI – National Center for Biotechnology Information – returned 9 results.
- Regulations.gov – returned 955 documents.
- US Grants.gov – returned 157 grant references.
- USA.gov – returned over 10 pages of documents – does not list total number.
- WorldWideScience.org – 1,287 top results returned from 65,671 found in all sources.
As you can see, Google Scholar for US Gov search found 981 documents while regular Google for US Gov returned only 87 documents! These Google and Google Scholar results documents are found mostly on Dept of Energy and NASA websites that Google apparently does index parts of these sites. Note, the US DoE Energy.gov site returns 32,389 results. The DoE federated search site – WorldWideScinece.org – includes the DoE, NASA, and many other US plus over a hundred Foreign Government scientific search sites.
The Energy.gov results just highlight our point! Energy.gov is a Dept. of Energy Federated site that returned 35 times as many documents as Google Scholar. WorldWideScience.org is a newer, Dept. of Energy managed, federated search engine that ranked the top 1,287 documents out of 65,671 documents from 106 global databases containing these five keywords.
These differences in the number of results just show that what Google says about their Googlebot search engine is true – they do not index the US or other Government information hidden behind search box HTML Forms!
Accelerate your Search beyond Google – Try it yourself for FREE:
DiscoverUSGov.com makes finding relevant US Government databases quick and easy for your projects!
Pete Fenner, President
Search Real Fast
If you want to receive our Bi-weekly Comments on Search, Business News, Technology, & Patents then just submit your Name and E-mail address in this form and we will add you to our blog distribution list. You may unsubscribe at any time by using the Unsubscribe link at the bottom of the distribution email.