-
Notifications
You must be signed in to change notification settings - Fork 62
/
README.md.generated
143 lines (97 loc) · 9.84 KB
/
README.md.generated
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
# Awesome CSV [](https://awesome.re)
**A carefully curated list of CSV-related tools and resources**
[CSV](https://en.wikipedia.org/wiki/Comma-separated_values) remains the most futuristic data format from the distant past.
XML has risen and fallen. JSON is just a flash in the pan. YAML is a poisoned chalice. **CSV will outlast them all.**
When the final cockroach breathes her last breath, her dying act will be to scratch her date of death in a CSV file for posterity.
## Contents
Here are some awesome tools for dealing with CSV:
## Tools
- [NimbleText/Live](https://NimbleText.com/Live) - Use patterns to manipulate CSV; the world's simplest code generator *.
- [PapaParse](https://www.papaparse.com) - A powerful in-browser CSV parser.
- [d3-dsv](https://github.com/d3/d3-dsv) - d3.js parser and formatter module for delimiter-separated values.
- [CSVKit](http://csvkit.readthedocs.org/en/0.7.3/) - CSV utilities that includes csvsql / csvgrep / csvstat and more.
- [XSV](https://github.com/BurntSushi/xsv) - A fast CSV command-line toolkit written in Rust.
- [sed (gnu tool)](https://www.gnu.org/software/sed/manual/sed.html) - Stream editor.
- [gawk (gnu tool)](https://www.gnu.org/software/gawk/manual/gawk.html) - Text processing and data extraction using [awk](http://pubs.opengroup.org/onlinepubs/009695399/utilities/awk.html).
- [awk by example](https://github.com/learnbyexample/Command-line-text-processing/blob/master/gnu_awk.md#default-field-separation) - Comprehensive examples of using awk.
- [Miller](http://johnkerl.org/miller/doc/) - Like sed / awk / cut / join / sort etc for name-indexed data such as CSV.
- [ParaText](https://github.com/wiseio/paratext) - CSV parsing at 2.5 GB per second.
- [CSVGet](http://github.com/fizx/csvget/tree/master) - Get structured data from sites as CSV.
- [CSVfix](https://code.google.com/p/csvfix/) - A tool for manipulating CSV data.
- [Tad](https://www.tadviewer.com) - A fast free cross-platform CSV viewer.
- [Nvd3-tags](http://blog.tryolabs.com/2015/02/27/nvd3-tags-a-tiny-library-for-making-charts-from-csv-data/) - A tiny library for making charts from csv data.
- [Powershell: Import-CSV](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/import-csv) - Powerful in-built facility for dealing with CSV, ([example](https://gist.github.com/dfinke/786ba9edae1b0265ada10b36a7a11ba9)).
- [CSV Tools](https://onlinecsvtools.com/) - A collection of useful CSV utilities.
- [graph-cli](https://github.com/mcastorina/graph-cli) - Flexible command line tool to create graphs from CSV data.
- [CSV to SQL](http://www.convertcsv.com/csv-to-sql.htm) - Online tool to create insert/update/delete etc from CSV data.
- [C#: kbCSV](https://github.com/kentcb/KBCsv/blob/master/README.md) - An efficient, easy to use .NET parsing and writing library for CSV.
- [csvprintf](https://github.com/archiecobbs/csvprintf) - UNIX command line utility for parsing and formatting output based on CSV files.
- [Mockaroo](https://www.mockaroo.com/) - Random data generator for CSV / JSON / SQL / Excel.
- [Ron's CSV Editor](https://www.ronsplace.eu/products/ronseditor) - Handles big files, does miraculous things. A timeless editor for a timeless format.
- [Rainbow CSV plugins](https://github.com/mechatroner/rainbow_csv#rainbow-csv-in-other-editors) - Collection of text editor plugins for CSV/TSV syntax highlighting. Available for [Vim](https://github.com/mechatroner/rainbow_csv), [VS Code](https://marketplace.visualstudio.com/items?itemName=mechatroner.rainbow-csv), [Atom](https://atom.io/packages/rainbow-csv), [Sublime Text](https://packagecontrol.io/packages/rainbow_csv) and other editors.
- [Mighty Merge](https://mightymerge.io/) - Join/union CSV files.
- [VisiData](https://www.visidata.org/) - Interactive, powerful, high-throughput csv explorer/manipulator, right in your terminal.
## Repair or Validate CSV
- [Csvlint.go](https://github.com/Clever/csvlint) - Command line tool for validating CSV files against RFC 4180.
- [csvstudio](http://www.csvstudio.com) - A smart app to repair syntax errors in very large CSV files.
- [scrubcsv](https://github.com/faradayio/scrubcsv) - Remove bad records from a CSV file and normalize (requires rust).
- [reconcile-csv](https://github.com/OpenRefine/reconcile-csv/blob/master/README.md) - Find relationships between a set of related CSVs.
## Generate Table Schema
> Wanted: More tools in this category.
- [CSV Schema](https://csv-schema.surge.sh/) - Analyzes a CSV file and generates database table schema, all within the browser.
## Treat CSV as SQL
- [TextQL](http://dinedal.github.io/textql/) - Execute SQL against CSV or TSV.
- [Datasette Facets](https://simonwillison.net/2018/May/20/datasette-facets/) - Faceted browse and a JSON API for any CSV File or SQLite DB.
- [q](https://harelba.github.io/q/) - Run SQL directly on CSV files.
- [RBQL](https://rbql.org) - Rainbow Query Language, a SQL-like language with JavaScript or Python backend.
- [PSKit Query](https://github.com/dfinke/PSKit#sql-query) - Powershell module lets you run simple queries over objects, including imported with csv.
## Convert to or from CSV
- [CSV to Table](https://github.com/vividvilla/csvtotable) - Convert CSV files to searchable and sortable HTML table.
## CSV <-> JSON
- [Agnes](http://www.secretgeek.net/agnes/twoWay.html) - Two way Csv to Json **.
- [csv2json](https://www.csvjson.com/csv2json) - Online tool to convert your CSV or TSV formatted data to JSON and [vice versa](https://www.csvjson.com/json2csv).
- [csv-to-json](https://mango-is.com/tools/csv-to-json/) - Easy, privacy-friendly and offline-first online csv to json converter.
## Essays
> Once you've found the perfect data serialization file format, you stop looking
>
> [David Wengier](https://twitter.com/davidwengier/status/1159606464220000257)
- [Thinking about CSV](https://blog.datacite.org/thinking-about-csv/) - Martin Fenner.
- [In Praise of CSV](https://usopendata.org/2015/03/10/csv) - Waldo Jaquith.
- [Stop Rolling Your Own CSV Parser!](http://www.secretgeek.net/csv_trouble) - Leon Bambrick ***.
- [So You Want To Write Your Own CSV code?](http://thomasburette.com/blog/2014/05/25/so-you-want-to-write-your-own-CSV-code/) - Thomas Burette.
- [Falsehoods Programmers Believe About CSVs](https://donatstudios.com/Falsehoods-Programmers-Believe-About-CSVs) - Jesse Donat.
- [ASCII Delimited Text - Not CSV or TAB delimited text](https://ronaldduncan.wordpress.com/2009/10/31/text-file-formats-ascii-delimited-text-not-csv-or-tab-delimited-text/) - Ronald Duncan.
## Data
- [US Data.gov](https://catalog.data.gov/dataset?res_format=CSV) - 18789+ CSV datasets.
- [Australian Government Open Data](https://data.gov.au/dataset?res_format=CSV) - 2715+ CSV datasets.
- [Reference data in csv](https://datahub.io/collections/reference-data) - Easy-to-use reference data in CSV and JSON formats.
- [awesome-public-datasets](https://github.com/awesomedata/awesome-public-datasets) - A topic-centric list of high-quality open datasets in public domains.
## Conferences
- [csv,conf](https://csvconf.com/) - A community conference for data makers everywhere.
## Standards
> The wonderful thing about standards is that there are so many of them to choose from.<br />—(Possibly) Grace Hopper.
- [RFC 4180](https://tools.ietf.org/html/rfc4180) - 'Common format and MIME Type for Comma-Separated Values (CSV) Files'.
- [Definition of the CSV Format](https://tools.ietf.org/html/rfc4180#section-2) - Definition.
- [MIME Type Registration of text/csv](https://tools.ietf.org/html/rfc4180#section-3) - Mime-type.
- [W3C: Model for Tabular Data and Metadata on the Web](https://www.w3.org/TR/tabular-data-model/) - W3C model.
- [CSV Schema Language](http://digital-preservation.github.io/csv-schema/csv-schema-1.2.html) - A language for defining and validating CSV data.
- [csvz](https://github.com/secretGeek/csvz) - Emerging standard for zipped csv containers with optional meta data ****.
- [csv,specs](https://github.com/csvspecs) - Comma-Separated Values (CSV) Format Specifications (and Tests) incl. CSV v1.0, CSV v1.1, CSV Strict, CSV <3 Numerics, CSV<3 JSON, CSV <3 YAML.
- [Tabular Data Resource](http://frictionlessdata.io/specs/tabular-data-resource/) - A [Data Resource](http://frictionlessdata.io/specs/data-resource/) specialized for describing tabular data like CSV files or spreadsheets.
- [CSVY](https://csvy.org/) - A standard for adding a YAML header to CSV files to describe their format.
## META: Other similar lists
- [structured-text-tools](https://github.com/dbohdan/structured-text-tools) - List of command line tools for manipulating CSV / XML / HTML / JSON / INI etc.
- [META-META](https://raw.githubusercontent.com/secretGeek/AwesomeCSV/master/awesomecsv.csv) - **This list as CSV**.
- [META-META-META](https://nimbletext.com/Live/-971009575/) - A NimbleText pattern that produces this markdown page from this list as a CSV.
## Code of Conduct
See [Code of Conduct](code-of-conduct.md)
## Funtribute
To experience the fun of contributing, see [Contributing](contributing.md)
## Footnotes
`*` <span id='footnote1' ></span> I'm the author of [NimbleText](https://NimbleText.com/Live). Of course I put it first on the list. If I didn't personally rate it I wouldn't have spent so much time making and improving it.
`**` <span id='footnote2' ></span> I wrote `agnes` but don't really endorse it for others to use (thus haven't migrated the source code to GitHub). It's slow and non-streaming. I'd go with `papa-parse`. On the plus side, `agnes` has a more comprehensive test suite and simpler api than most.
`***` <span id='footnote3' ></span> Mine too.
`****` <span id='footnote4' ></span> Again.
## License
[](https://creativecommons.org/publicdomain/zero/1.0/)
To the extent possible under law, [Leon Bambrick](http://secretgeek.net) has waived all copyright and related or neighboring rights to this work.