EVE Technology Lab

 
^ Back to top

Topic is locked indefinitely.

 

Odyssey 1.0.12 89967 SDE Conversions - Mysql, XLS/CSV, Postgres, sqlite

Author
Vote Steve Ronuken for CSM
#1 Posted: 2013.07.03 13:15  |  Edited by: Steve Ronuken
#2 Posted: 2013.07.03 14:51
Thank you sir!
Gallente Federation
#3 Posted: 2013.07.03 20:34
Yay! Postgres!
Beyond Event Horizon.
#4 Posted: 2013.07.04 19:20
Ivy League
#5 Posted: 2013.07.07 09:28
Thank you!
Download is the meaning of life, upload is the meaning of intelligent life
http://eve.nikr.net - home of jEveAssets
Fraternity.
#6 Posted: 2013.07.17 11:24
The exported mysql translation table become unreadable. It's works fine before.
Vote Steve Ronuken for CSM
#7 Posted: 2013.07.17 12:06
Zanto Snix wrote:
The exported mysql translation table become unreadable. It's works fine before.


Looks like it's been screwed up by a change in the process I was using to load the data (which cut out several steps).


UTF8 conversions can be painful at times.

I'll kick at it the next time I convert.
Rooks and Kings
#8 Posted: 2013.07.18 13:40
Zanto Snix wrote:
The exported mysql translation table become unreadable. It's works fine before.



try this it works for me
1) mysqldump -h -u -p --default-character-set=latin1 --skip-set-charset NAME_DUMP_DB trntranslations > trntranslations.sql
2) mysql -h -u -p --default-character-set=utf8 NAME_DUMP_DB < trntranslations.sql


Fraternity.
#9 Posted: 2013.07.19 01:54
thanks you Preck, it works like a charm.
Ivy League
#10 Posted: 2013.07.23 09:43
@Steve Ronuken
FYI the invtypes table is also affected by the UTF8 conversions problem:

Please try the following queue on your MySQL export:
SELECT * FROM invtypes WHERE typeName LIKE '%€%' OR typeName LIKE '%¬%'

For me that returns some rows with bad characters.
TypeID: 30952, 32371, 32372, 32383, 32398, 32407, 33041, 33046, 33048 (there could be more, I missed)

It would be awesome if you released a fixed version Blink

Anyway, thank you for all your hard work! :)
Download is the meaning of life, upload is the meaning of intelligent life
http://eve.nikr.net - home of jEveAssets
Vote Steve Ronuken for CSM
#11 Posted: 2013.07.23 09:46
I'll look into it, when my server's settled down.

Recently migrated to a new box, and it appears to be having some trouble. Just put in a 'check the hardware' ticket on it.
Ivy League
#12 Posted: 2013.07.23 11:22
@Steve Ronuken
Good luck with your new server....

I look forward to a fixed release.
Thank you for doing it - you're awesome Blink
Download is the meaning of life, upload is the meaning of intelligent life
http://eve.nikr.net - home of jEveAssets
Vote Steve Ronuken for CSM
#13 Posted: 2013.07.23 22:14  |  Edited by: Steve Ronuken
Just the sql files this time:
https://www.fuzzwork.co.uk/dump/odyssey-1.0.12-89967-fixed/


I've tested them, and I'm seeing the russian and the japanese characters showing up now, both in the base, and the import from the export file.


And if you'll excuse me, I'm going to sit in a corner now, and rock back and forth, muttering about utf8



Yes, this had been screwed up earlier, losing a lot of date in the conversion process. I'm still working on a way to do the conversion that doesn't depends on an ancient bit of software that's a pita to get working.
#14 Posted: 2013.08.14 15:35
Thank you for these. You are probably sick of me... but a year ago I used to import these as so:

mysql.exe --user=root --password=xxxxx test < agtAgentTypes.sql

but that now returns an error. What is the method for importing these?
(I see the files are no longer plan text ascii, but binary)
I extracted the file using 7zip on win 8
Thanks!
Vote Steve Ronuken for CSM
#15 Posted: 2013.08.14 15:54
grr. Something's happening with 7zip with those files. It's not actually unzipping them when you tell it to. If you extract the file, rename it to a .gz, and open it again, you can get at the details. I didn't bother testing them, as they'd never been trouble in the past.

I suspect it's related to the .utf8 in the filename, but that's a wild guess.

I've unzipped then bzip2ed them all, and I can open it using 7zip, without needing to mess with them.

#16 Posted: 2013.08.14 16:08
Ah the old gzip in a gzip trick. Ok yeah that worked - thanks.

mysql55-odyssey-1.0.12-89967.tgz imported fine.

Is it UTF8 'fixed' or should I import the individual sql files?
(or do I care as I run everything in EN).
Vote Steve Ronuken for CSM
#17 Posted: 2013.08.14 16:58
6ie wrote:
Ah the old gzip in a gzip trick. Ok yeah that worked - thanks.

mysql55-odyssey-1.0.12-89967.tgz imported fine.

Is it UTF8 'fixed' or should I import the individual sql files?
(or do I care as I run everything in EN).



It did affects a little in invTypes, when extended characters were used. (look at, umm, 2072, using a utf8 capable client. Should have an em-dash in it, between queen and bright)

But mostly it was trnTranslations.
#18 Posted: 2013.08.14 18:05
yes, using invTypes.utf8.sql fixed the funny chars that mysql55-odyssey-1.0.12-89967.sql inserted - thanks.

When opening invTypes.utf8.sql in notepad++, it says it is ANSI encoded.
Shouldn't you be creating them as UTF8 encoded?
Vote Steve Ronuken for CSM
#19 Posted: 2013.08.14 20:31
They are UTF8 encoded. Just not UTF8 with BOM. (at least, I'm pretty sure this is the case. If it's not, oops?)

Without a BOM, it's very difficult to determine if a text file is UTF8 or not.


http://stackoverflow.com/questions/4520184/how-to-detect-the-character-encoding-of-a-text-file



Forum Jump