Data Wrangling Blog - Latest Comments in Some Datasets Available on the Web

Re: Some Datasets Available on the Web

Yuri Burger — Mon, 18 Nov 2013 10:36:52 -0000

one more link: http://endb-consolidated.ai... random 10,000 worldwide companies sampled from HitCompanies (all data in this DB extracted and updated automatically from WWW using AI and Machine Learning): company name and aliases, company description, industry tags, industry codes, registration numbers, addresses, phone numbers, VAT numbers, website, number of about/contact/management/product pages, incorporation date, team size, number of clients and partners, number of emails, number of key changes (client/partner changes, contact changes, people changes), and many more.

Re: Some Datasets Available on the Web

Modarres Zadeh — Tue, 09 Aug 2011 07:38:02 -0000

Where can i find example datasets for inner workings of an insurance company or bank ?

Re: Some Datasets Available on the Web

Sanjay Kushwaha — Tue, 09 Aug 2011 06:12:39 -0000

can any one help me to get a dataset which can describe the effort estimation of web project including the attributes
TypeProj- Categorical Type of project (new or enhancement).nLang- Ratio Number of different development languages used.DocProc -Categorical If project followed defined and documented process.ProImpr- Categorical If project team involved in a process improvement programme.Metrics -Categorical If project team part of a software metrics programme.DevTeam- Ratio Size of a project’s development team.TeamExp -Ratio Average team experience with the development language(s) employed.TotEffort -Ratio Actual total effort in person hours used to develop an application.EstEffort -Ratio Estimated total effort in person hours to develop an application.Accuracy -Categorical Procedure used to record effort data.

Re: Some Datasets Available on the Web

Vinoth — Fri, 05 Feb 2010 15:50:47 -0000

hi
can anyone tell me how i can obtain data sets for movie ratings to import into WEKA

Re: Some Datasets Available on the Web

manoj — Fri, 01 Jan 2010 01:03:12 -0000

i need Boolean dataset for association mining for my MTech project.Please provide me the address for the same. It will be a great help

Re: Some Datasets Available on the Web

endah — Tue, 15 Dec 2009 21:10:07 -0000

I need dataset image fingerprint for free,,,where I can find it?
thank u

Re: Some Datasets Available on the Web

Mandy — Wed, 23 Sep 2009 21:47:29 -0000

I need RFID Supply Chain data for my research. Where can I find it??

Re: Some Datasets Available on the Web

rajeswari — Wed, 15 Jul 2009 06:39:22 -0000

this is rajeswari.,
i am doing a project in association rule mining in datamining for time related data. so i need a dataset with time related data ie,, temporal data.so please send me time related data.it's really helpful for my data

Re: Some Datasets Available on the Web

halim — Mon, 13 Jul 2009 20:43:46 -0000

Dear all,

where can i find the dataset for manufacturing?? such as 'defect' or 'not defect' prediction..please ..help me..i am in urgent condition...thx b4

Re: Some Datasets Available on the Web

uma — Fri, 10 Jul 2009 04:22:01 -0000

the site u mentioned contains the required data for my project.i thank u for this great help

Re: Some Datasets Available on the Web

Pete Skomoroch — Wed, 08 Jul 2009 03:33:11 -0000

The FDA has some data like that:

http://www.fda.gov/Drugs/Gu...

Some analogous time series data might be worth looking at as well:

http://delicious.com/pskomo...

Re: Some Datasets Available on the Web

uma — Wed, 08 Jul 2009 01:48:29 -0000

where can i get dataset for mining unexpected temporal association rules(eg:application in adverse drug reaction)

Re: Some Datasets Available on the Web

Ken — Fri, 13 Feb 2009 18:11:49 -0000

A personal favorite: ITRDB: ftp://ftp.ncdc.noaa.gov/pub...

Re: Some Datasets Available on the Web

Peter Skomoroch — Mon, 10 Nov 2008 20:49:53 -0000

Tim,

I like the site, and the capability to download in various formats. A REST or soap API would be nice, or at least an index page for each format with direct paths to the individual downloads.

-Pete

Re: Some Datasets Available on the Web

Tim — Mon, 10 Nov 2008 20:33:01 -0000

what do you think of http://data.un.org ?

Re: Some Datasets Available on the Web

Rufus Pollock — Thu, 10 Apr 2008 05:29:06 -0000

I don't know whether you've seen CKAN (Comprehensive Knowledge Archive Network). This is a project started by the Open Knowledge Foundation (of which I'm a part) and was launched about a year ago and seeks to perform exactly the type of registry task you've started upon here (though limited to open material only). As the blurb on the front-page says:

CKAN is the Comprehensive Knowledge Archive Network, a registry of open knowledge packages and projects (and a few closed ones). CKAN is the place to search for open knowledge resources as well as register your own – be that a set of Shakespeare's works, a global population density database, the voting records of MPs, or 30 years of US patents.

Those familiar with freshmeat or CPAN can think of CKAN as providing an analogous service for open knowledge.

Re: Some Datasets Available on the Web

Lili — Wed, 09 Apr 2008 19:32:14 -0000

What about including the wiki http://www.numberzoom.com/ which is a user-contributed phone numbers database. It's mosty reverse Caller ID for looking up what telemarketers or collection agencies have called, but there is no reason why other numbers wouldn't be on the site.

Re: Some Datasets Available on the Web

Peter Skomoroch — Wed, 09 Apr 2008 15:37:26 -0000

Brent sorry I missed that, this data will be useful for some identity matching projects I'm testing. I just found the programmable web description of your api as well: http://www.programmableweb.com/api/cogmap

Re: Some Datasets Available on the Web

Brent — Wed, 09 Apr 2008 15:29:03 -0000

It's in there! http://www.cogmap.com/blog/2008/03/04/cogmap-apis/

-- brent

Re: Some Datasets Available on the Web

Peter Skomoroch — Wed, 09 Apr 2008 14:24:48 -0000

Brent,

Just added Cogmap to my dataset bookmarks... any chance on releasing a raw dataset or REST api to fetch raw orgchart data?

-Pete

Re: Some Datasets Available on the Web

Brent — Wed, 09 Apr 2008 13:50:48 -0000

The omission of cogmap makes me sad! Cogmap provides organization chart data for thousands of companies and exposes it all through a variety of web services.

Re: Some Datasets Available on the Web

Guest — Sat, 19 Jan 2008 11:50:21 -0000

Looks like Google is going to start providing access to loads of open sourced data sets (http://blog.wired.com/wired....

Re: Some Datasets Available on the Web

Peter Skomoroch — Sat, 19 Jan 2008 08:50:14 -0000

skj,

The WebBase Project link includes some chat data. It would be pretty easy to crawl for that data, provided terms of use for the chat sites are followed. Here is a recent list of hosts Stanford WebBase crawled, which includes chat sites (this link might not be permanent):

http://dbpubs.stanford.edu:8090/~testbed/doc2/WebBase/crawl_lists/crawled_hosts.0403

Re: Some Datasets Available on the Web

Peter Skomoroch — Sat, 19 Jan 2008 08:35:17 -0000

civilian,

The LDC site was up yesterday. It may have been hammered by reddit/del.icio.us users? I think some of the datasets they have are extremely large (for example the google N-grams), so there is a handling fee for non-commercial researchers. As far as commercial use fees, many data providers restrict use entirely. Open access to more data would be great ... except where privacy issues are involved. Sometimes there are also competitive reasons for restrictive licenses.

See more on the issues here:

http://en.wikipedia.org/wiki/Open_data
http://en.wikipedia.org/wiki/Data_privacy

related discussion:

http://news.ycombinator.com/item?id=100197

Re: Some Datasets Available on the Web

skj — Sat, 19 Jan 2008 05:44:20 -0000

Are there any datasets of chat logs? Chat conversations (from IRC or otherwise)?