What am I trying to answer? What institutions could answer the question?

• If outdated, go to the source and contacting the department goes so far. Most is just poorly formatted for online. Also ask for what requests they get. Always get sample data first. Give them the entities you want samples for Are you going to be locked into the set or can you off ramp? How long have they been around for?

Keywords to include in your search

• Data hub

Framing the search

• Key entities involved
• Gradunarility (insights/statistics, agregated, tied to atomic units)
• Quality
• Recency
• Provenance and licensing - Was it legally sourced?

Finding Aggregators

• Best to talk them directly about what you need because some things mayb be off the venue or may be referred to differently
• CoStar in real estate

Open Data

• data.gov
• Google dataaset Search
• Data Refuge - More niche but good for vanishing open data. Eg. EPA records that disappear from the internet. Run by awesome volunteer teams that do hackathons to maintian
• Data.world - Less open data than before but still hundreds of thousands available
• Internet Archive - All the TV Chirons that have been on every news story
• Our World in Data
• Other statistic aggregators

Semi-open Data

• Person leading research group
• Muckrock
• FOIA - Specifically for the schema of the database. Sometimes you may have to pay for it
• Investigative Reporters and Editors -
  • Has annual conference like NICAR

Blended Open and Paid

• ProPublica - Marketplace for data, also includes cleaned versions of public data sets
• Statista -

Piad Data

• AWS Marketplace
• Datarade
• Snowflake -
• Nomad data - Data matchmaker
• Eagle Alpha - Data matchmaker

Creating your own

• YouGov - Designing surveys

Slow people down by submitting FOIA