EVE Technology Lab

 
  • Topic is locked indefinitely.
 

Odyssey 1.0.12 89967 SDE Conversions - Mysql, XLS/CSV, Postgres, sqlite

Author
Vote Steve Ronuken for CSM
#1 - 2013-07-03 13:15:58 UTC  |  Edited by: Steve Ronuken
#2 - 2013-07-03 14:51:38 UTC
Thank you sir!
Gallente Federation
#3 - 2013-07-03 20:34:57 UTC
Yay! Postgres!
Beyond Event Horizon.
#4 - 2013-07-04 19:20:23 UTC
Ivy League
#5 - 2013-07-07 09:28:45 UTC
Thank you!

Download is the meaning of life, upload is the meaning of intelligent life http://eve.nikr.net - home of jEveAssets

Amarr Empire
#6 - 2013-07-17 11:24:26 UTC
The exported mysql translation table become unreadable. It's works fine before.
Vote Steve Ronuken for CSM
#7 - 2013-07-17 12:06:36 UTC
Zanto Snix wrote:
The exported mysql translation table become unreadable. It's works fine before.


Looks like it's been screwed up by a change in the process I was using to load the data (which cut out several steps).


UTF8 conversions can be painful at times.

I'll kick at it the next time I convert.

Woo! CSM 9!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Rooks and Kings
#8 - 2013-07-18 13:40:38 UTC
Zanto Snix wrote:
The exported mysql translation table become unreadable. It's works fine before.



try this it works for me
1) mysqldump -h -u -p --default-character-set=latin1 --skip-set-charset NAME_DUMP_DB trntranslations > trntranslations.sql
2) mysql -h -u -p --default-character-set=utf8 NAME_DUMP_DB < trntranslations.sql


http://eve-universe.eu

Amarr Empire
#9 - 2013-07-19 01:54:38 UTC
thanks you Preck, it works like a charm.
Ivy League
#10 - 2013-07-23 09:43:59 UTC
@Steve Ronuken
FYI the invtypes table is also affected by the UTF8 conversions problem:

Please try the following queue on your MySQL export:
SELECT * FROM invtypes WHERE typeName LIKE '%€%' OR typeName LIKE '%¬%'

For me that returns some rows with bad characters.
TypeID: 30952, 32371, 32372, 32383, 32398, 32407, 33041, 33046, 33048 (there could be more, I missed)

It would be awesome if you released a fixed version Blink

Anyway, thank you for all your hard work! :)

Download is the meaning of life, upload is the meaning of intelligent life http://eve.nikr.net - home of jEveAssets

Vote Steve Ronuken for CSM
#11 - 2013-07-23 09:46:16 UTC
I'll look into it, when my server's settled down.

Recently migrated to a new box, and it appears to be having some trouble. Just put in a 'check the hardware' ticket on it.

Woo! CSM 9!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Ivy League
#12 - 2013-07-23 11:22:20 UTC
@Steve Ronuken
Good luck with your new server....

I look forward to a fixed release.
Thank you for doing it - you're awesome Blink

Download is the meaning of life, upload is the meaning of intelligent life http://eve.nikr.net - home of jEveAssets

Vote Steve Ronuken for CSM
#13 - 2013-07-23 22:14:48 UTC  |  Edited by: Steve Ronuken
Just the sql files this time:
https://www.fuzzwork.co.uk/dump/odyssey-1.0.12-89967-fixed/


I've tested them, and I'm seeing the russian and the japanese characters showing up now, both in the base, and the import from the export file.


And if you'll excuse me, I'm going to sit in a corner now, and rock back and forth, muttering about utf8



Yes, this had been screwed up earlier, losing a lot of date in the conversion process. I'm still working on a way to do the conversion that doesn't depends on an ancient bit of software that's a pita to get working.

Woo! CSM 9!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

#14 - 2013-08-14 15:35:24 UTC
Thank you for these. You are probably sick of me... but a year ago I used to import these as so:

mysql.exe --user=root --password=xxxxx test < agtAgentTypes.sql

but that now returns an error. What is the method for importing these?
(I see the files are no longer plan text ascii, but binary)
I extracted the file using 7zip on win 8
Thanks!
Vote Steve Ronuken for CSM
#15 - 2013-08-14 15:54:06 UTC
grr. Something's happening with 7zip with those files. It's not actually unzipping them when you tell it to. If you extract the file, rename it to a .gz, and open it again, you can get at the details. I didn't bother testing them, as they'd never been trouble in the past.

I suspect it's related to the .utf8 in the filename, but that's a wild guess.

I've unzipped then bzip2ed them all, and I can open it using 7zip, without needing to mess with them.

Woo! CSM 9!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

#16 - 2013-08-14 16:08:47 UTC
Ah the old gzip in a gzip trick. Ok yeah that worked - thanks.

mysql55-odyssey-1.0.12-89967.tgz imported fine.

Is it UTF8 'fixed' or should I import the individual sql files?
(or do I care as I run everything in EN).
Vote Steve Ronuken for CSM
#17 - 2013-08-14 16:58:30 UTC
6ie wrote:
Ah the old gzip in a gzip trick. Ok yeah that worked - thanks.

mysql55-odyssey-1.0.12-89967.tgz imported fine.

Is it UTF8 'fixed' or should I import the individual sql files?
(or do I care as I run everything in EN).



It did affects a little in invTypes, when extended characters were used. (look at, umm, 2072, using a utf8 capable client. Should have an em-dash in it, between queen and bright)

But mostly it was trnTranslations.

Woo! CSM 9!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

#18 - 2013-08-14 18:05:33 UTC
yes, using invTypes.utf8.sql fixed the funny chars that mysql55-odyssey-1.0.12-89967.sql inserted - thanks.

When opening invTypes.utf8.sql in notepad++, it says it is ANSI encoded.
Shouldn't you be creating them as UTF8 encoded?
Vote Steve Ronuken for CSM
#19 - 2013-08-14 20:31:06 UTC
They are UTF8 encoded. Just not UTF8 with BOM. (at least, I'm pretty sure this is the case. If it's not, oops?)

Without a BOM, it's very difficult to determine if a text file is UTF8 or not.


http://stackoverflow.com/questions/4520184/how-to-detect-the-character-encoding-of-a-text-file



Woo! CSM 9!

Fuzzwork Enterprises

Twitter: @fuzzysteve on Twitter

Forum Jump