sqlite import / export util supports csv, json, html, sql and xml

User projects written in or related to FreeBASIC.
Post Reply
thrive4
Posts: 72
Joined: Jun 25, 2021 15:32

sqlite import / export util supports csv, json, html, sql and xml

Post by thrive4 »

description
A 'meat and potato' utility that allows for
- export of data
|sqlite database via sql query or a folder plus filespec
|to csv, json, html(table), sql and xml
- import of data to a sqlite database / table via
|csv, json and xml via conversion to sql.

Additionally it has special support for .mp3 or .jpg
meta information extraction this will extract the tag info
and or create media links in the generated html table,
a checkfile function which verifies if file excists
according to <path and file> in specified field specified
in a sql query.

Other basic features are:
- basic info database and tables
|cmdsqlite.exe [dbname] showtables
|displays table names contained in [dbname]
|cmdsqlite.exe [dbname] showfields [tablename]
|displays fieldnames contained in [tablename]
- minimal configuration / localization via .ini files

From a code perspective the central trick / conceit is
using a 'spindle' aka key / value array a method which
has it's shortcomings but scales to a degree...

Intended use is mostly educational, many rough edges
with regard to validation of data, etc but might
be useful to some as bare bones conversion util.

usage
cmdsqlite.exe [dbname] [query] optional [exporttype]
cmdsqlite.exe [folder] [filespec] [exporttype]
cmdsqlite.exe [dbname] [query] checkfile [field name]
checkfile returns -1 if the file exists, otherwise zero 0
cmdsqlite.exe [dbname] showtables
cmdsqlite.exe [dbname] showfields [tablename]

options
via \conf\conf.ini\
[output]
' extra sets additional parsing per filetype mp3, jpg
' options: default, extra
htmloutput = extra

Tested and compiled on windows 7 / 10 as 32bit app

release (32bit windows)
https://github.com/thrive4/util.fb.cmdsqlite/releases

source
https://github.com/thrive4/util.fb.cmdsqlite

requirements
sqlite.dll 32-bit DLL (x86) for SQLite version 3.43.0.
https://www.sqlite.org/download.html

quick tutorial

https://github.com/thrive4/util.fb.cmds ... torial.txt

special thanks

tips on commandline parsing via:
viewtopic.php?t=31889 code by coderJeff
data set vgsales via:
https://gist.github.com/zhonglism/f146a ... 26451f841e
thrive4
Posts: 72
Joined: Jun 25, 2021 15:32

Re: sqlite import / export util supports csv, json, html, sql and xml

Post by thrive4 »

Quite a chuncky update added and fixed the following:

update v1.1 02/11/2023
- added export meta info mp3 to other fileformats
| csv, json, sql and xml
- added export meta info mp3 album to html
- added filter table javascript
| courtesy to: https://www.w3schools.com/howto/howto_j ... _table.asp
| partially implemented filter on specific field
- replaced sort routine for tables javascript
| significant improvements in speed and number
| of rows that can be handeled by browser
| many thanks to tofsjonas
| see: https://github.com/tofsjonas/sortable
- modified main.css less cluterd html table and other style tweaks
- fixed json export double first fieldname and value
| with datasource as folder plus filespec
- improved xml export sanitize values ampersand, etc
- improved csv import some what better handling
| of ".." (doubleqoute enclosed) fields
| remove possible trailing comma at end of record
| added rather strict cleaning of header todo needs more work....
- fixed deselect issue audio playing button html table
- fixed log creation (missing message)
- fixed appname made truncation .exe optional
- fixed checkpath did not set correct path after
| function exit when path is correct
- corrected isalphanumeric function
- misc fixes to getmp3cover
- edits to help.ini all languages
- added initial example with wikidata (sparql) as datasouce
|see data/wikidata.nfo (csv | json) xml is... problematic ...

See:
https://github.com/thrive4/util.fb.cmds ... daa524a30e
thrive4
Posts: 72
Joined: Jun 25, 2021 15:32

Re: sqlite import / export util supports csv, json, html, sql and xml

Post by thrive4 »

Integrated previously posted functionality from:
viewtopic.php?p=296549
'barebone export mp3 cover (jpeg / png) no dll's'
viewtopic.php?p=288025
'mht 2 html'

update v1.2 18/12/2023
- integrated mp3 album cover export
| from https://github.com/thrive4/util.fb.mp3cover
| added filesize and orientation image as field data
| to mp3cover.csv report
- integrated .mht convert (converts .mht to .html and .txt)
| from viewtopic.php?p=288025
- updated wordwrap2file some what better text formatting
- added sticky table header for data tables
| sticky tip courtesy https://codepen.io/stephen-george-west/pen/rNpjXYL
- updated help.ini with new functionality
- partial implementation json data format escaping rules

See: https://github.com/thrive4/util.fb.cmdsqlite
Post Reply