'New' D-Mod File Format
Installing D-Mods is a complete pain in the ass, especially for those who aren't computer gurus.
That is why I recommend creating a 'new' file format called ".dmod". With a proper installer, a Dinker could just double-click on the .dmod file, and the D-Mod would automatically install. Patches with a .dmod extension would automatically overwrite existing files in the D-Mod directory.
However, it would be foolish to actually create a new compressable file format for installing D-Mods. So, my idea is to just have it be a standard zip file. The extension could even be something like .zip.dmod so people would know you could open it in a zip program.
Anybody else think this is a good idea?
That is why I recommend creating a 'new' file format called ".dmod". With a proper installer, a Dinker could just double-click on the .dmod file, and the D-Mod would automatically install. Patches with a .dmod extension would automatically overwrite existing files in the D-Mod directory.
However, it would be foolish to actually create a new compressable file format for installing D-Mods. So, my idea is to just have it be a standard zip file. The extension could even be something like .zip.dmod so people would know you could open it in a zip program.
Anybody else think this is a good idea?
Seems like a good idea, though M*rr*w*nd players don't seem to mind... A couple other thoughts:
I take it this there would be some kind of program to do with this? So wouldn't finding a good library or plug-in of some to handle the unompression be an important part of selecting a compression standard?
While we're thinking about that, .zip isn't one of the better ones in terms of size, as long as people have to download a special program, it seems to me to make sense to select one that has better compression.
I take it this there would be some kind of program to do with this? So wouldn't finding a good library or plug-in of some to handle the unompression be an important part of selecting a compression standard?
While we're thinking about that, .zip isn't one of the better ones in terms of size, as long as people have to download a special program, it seems to me to make sense to select one that has better compression.
uhm, why dont we just zip up the files, and make a executable zipped file, its the most easiest and fastest way.
When you open it (you dont need winzip) the install directory is automaticly set to the authors directory. (standard dir: c:\program files\dink smallwood\dmod)
or just use install maker, like im gonna do soon
When you open it (you dont need winzip) the install directory is automaticly set to the authors directory. (standard dir: c:\program files\dink smallwood\dmod)
or just use install maker, like im gonna do soon
Install maker and executable zip files are ok... if you installed Dink in the default directory. If you didn't, you're kinda screwed.
And it also assumes you're using a windows machine. With the port of Dink to Linux and other platforms pending, neither of these are a good idea.
And it also assumes you're using a windows machine. With the port of Dink to Linux and other platforms pending, neither of these are a good idea.
I take it this there would be some kind of program to do with this? So wouldn't finding a good library or plug-in of some to handle the unompression be an important part of selecting a compression standard?
Yeah, I assume there are some simple uncompression libraries for C++.
it seems to me to make sense to select one that has better compression.
Hmm, true. .rar.dmod then?
Yeah, I assume there are some simple uncompression libraries for C++.
it seems to me to make sense to select one that has better compression.
Hmm, true. .rar.dmod then?
Hmm, I never really thought about this before. However, this is a very good idea (*Thinks of Unreal Tournament's level style*).
I suppose you are suggesting something like a FastFile? In order to avoid sinking that low and to support the widest variety of platforms possible, I would suggest either a zlib-based file, or a bzlib-based file (which generally offers a better compression ratio). Another reason for this is to avoid closed-source compression algorithms as I believe (correct me if I'm wrong) that both .ZIP and .RAR are both proprietary formats.
If agreed as a good idea, I'll put this on my list of things to do after I implement the Directory-crawling infrastructure that SO pwns! I'm just a lazy guy, I guess.
EDIT: Could somebody explain this "M*rr*w*nd" thing to me?
I suppose you are suggesting something like a FastFile? In order to avoid sinking that low and to support the widest variety of platforms possible, I would suggest either a zlib-based file, or a bzlib-based file (which generally offers a better compression ratio). Another reason for this is to avoid closed-source compression algorithms as I believe (correct me if I'm wrong) that both .ZIP and .RAR are both proprietary formats.
If agreed as a good idea, I'll put this on my list of things to do after I implement the Directory-crawling infrastructure that SO pwns! I'm just a lazy guy, I guess.
EDIT: Could somebody explain this "M*rr*w*nd" thing to me?
Essentially, *any* compression algorithm would work, just as long as you could open it in any standard zip program. That way it doesn't *require* you to have any extra software, and you can open it up and extract separate files without installing the whole thing.
Are zlib and bzlib variations of tar files?
M*rr*w*nd = Morrowind
Are zlib and bzlib variations of tar files?
M*rr*w*nd = Morrowind
Well, you're right, but off the top of my head, rar isn't available for linux and the DynaZIP un/compression routines are not available as a library. Someone correct me on this if I'm wrong (DesPlesda? ewe2?).
Now, zlib is the library that powers the gzip files, bzlib is the library derived from zlib that powers the bzip2 files, and tar is a whole nother story. The three, .gz, .bz2, and .tar are three completely different formats. Tar was the first to come along (just look at the name: Tape ARchive), and was used to make backing up files to tape much easier. Now, the only thing tar is used for is an intermediary between a compression program: the ability to include more than one file in an archive (as .bz2 and .gz only support compression of one file). Now, tar sucked mainly for one reason: depending on what you were "compressing", the total size (c|w)ould INCREASE! Therefore, .gz was developed to compress that, and .bz2 as an even better alternative.
So, I'm suggesting the use of zlib and bzlib since they are avalible on Winblows. In fact, the current TDP engine currenly ((Not sure if that was even a pun (It was intended, but it wasn't intended to suck (the pun, I mean).)) <-- Ooh, double parenthetical notation!) uses zlib (becuase it uses libpng, which uses zlib).
And, I think I could figure out that M*rr*w*nd was the same as Morrowind, but my question was more along the lines of, "Will somebody tell me why everyone hates/is afraid of/is indifferent to Morrowind?"
Now, zlib is the library that powers the gzip files, bzlib is the library derived from zlib that powers the bzip2 files, and tar is a whole nother story. The three, .gz, .bz2, and .tar are three completely different formats. Tar was the first to come along (just look at the name: Tape ARchive), and was used to make backing up files to tape much easier. Now, the only thing tar is used for is an intermediary between a compression program: the ability to include more than one file in an archive (as .bz2 and .gz only support compression of one file). Now, tar sucked mainly for one reason: depending on what you were "compressing", the total size (c|w)ould INCREASE! Therefore, .gz was developed to compress that, and .bz2 as an even better alternative.
So, I'm suggesting the use of zlib and bzlib since they are avalible on Winblows. In fact, the current TDP engine currenly ((Not sure if that was even a pun (It was intended, but it wasn't intended to suck (the pun, I mean).)) <-- Ooh, double parenthetical notation!) uses zlib (becuase it uses libpng, which uses zlib).
And, I think I could figure out that M*rr*w*nd was the same as Morrowind, but my question was more along the lines of, "Will somebody tell me why everyone hates/is afraid of/is indifferent to Morrowind?"
Summer before last... DethLord would mention Morrowind in every message board post/review/moan of pleasure. So then, as a joke, I changed every 'dink' into 'Morrowind' on the board, and even changed TDN to The Morrowind Network or something. I forget.
Oh, and WC always got Uber Pissed whenever Morrowind was mentioned, hence my trouble making.
Edit: Hmm, is bz2 supported by most zip programs? It does seem rather nice. And Powerarchiver lists it under Tar, so
Oh, and WC always got Uber Pissed whenever Morrowind was mentioned, hence my trouble making.
Edit: Hmm, is bz2 supported by most zip programs? It does seem rather nice. And Powerarchiver lists it under Tar, so
I know of no unzipping programs that do not support .bz2 decompression, but few that support compression (one being PowerArchiver). Hmm, I'll look into that.
And for the people who have NO clue:
The Bzip2 and libbzip2 Homepage
The Gzip Homepage
The zlib Homepage
Some guy's blog of gzip versus bzip2
As noted in Jeremy Zawodny's blog above, gzip is a much better de/compressor for frequently used files. I will personally look into this more, but we may be better off with gzip. :/
And for the people who have NO clue:
The Bzip2 and libbzip2 Homepage
The Gzip Homepage
The zlib Homepage
Some guy's blog of gzip versus bzip2
As noted in Jeremy Zawodny's blog above, gzip is a much better de/compressor for frequently used files. I will personally look into this more, but we may be better off with gzip. :/
How do you figure that? For most people, it will be install once and delete. Not exactly a 'frequently used file'.
Edit: And it looks like Winzip supports gzip and not bz2... I wonder what Windows XP supports by default, hrm.
Edit: And it looks like Winzip supports gzip and not bz2... I wonder what Windows XP supports by default, hrm.
November 24th 2003, 12:52 PM
Beuc
My tests on DMods compressions showed that .tar.bz2 is better than .tar.gz - between 10% and 20% less.
.7z is even slightly better - check www.7-zip.org, it is free software and it also supports all mentioned formats except .ace.
Please use an open format so people do not have to get WinRAR/WinACE - usually most people can decode .rar, it is less common to extract .ace, but you just have to get the official proprietary and paying software to *encode* files in those formats. bz2 seems the best compromise quality/deployment.
I also really like the convention in all the .tar.* I saw to contain a carefully named directory containing all the files - it is better when you use the command-line, so you are sure never to extract everything in the current directory :/
Incidentally, it is not *that* hard to install a DMod.
.tar is not a compression format, just a portable archive one, and the _slightly_ increasing size is due to the additional information (filename, permissions...) that are usually stored in your FAT, just when you compress a .zip file witht the 'store'/'no compression' method -- and that's why you use gzip or bzip2 on the .tar archive to get what WinRAR would call a "solid" archive.
I think WXP only supports .zip files, but I do not have one installed here, so I cannot test.
Self-Extracting zip files are ok because you still can extract the .zip archive from inside it with an extractor - so you can avoid viruses and use it under GNU. However be sure not to use WinZip, but rather a free application (FreeZip, 7-Zip...), since you must get the registered version of WinZip to make a *legal* SFX archive.
Install Maker uses its own format and though you can use Wine to extract it, it is very tedious, so it's better avoid it.
Btw, if people are too dumb to install a DMod, I guess they are too dumb to change the default install directory.
The ending .dmod extension may cause some troubles on some servers/browsers, since they may display it instead of d/l-ing it. It's better create a drag&drop interface (eg improve CustomFE or any front-end), or drag&drop icon on the desktop (that in this case gets the filename using the application's commnad-line parameters).
.7z is even slightly better - check www.7-zip.org, it is free software and it also supports all mentioned formats except .ace.
Please use an open format so people do not have to get WinRAR/WinACE - usually most people can decode .rar, it is less common to extract .ace, but you just have to get the official proprietary and paying software to *encode* files in those formats. bz2 seems the best compromise quality/deployment.
I also really like the convention in all the .tar.* I saw to contain a carefully named directory containing all the files - it is better when you use the command-line, so you are sure never to extract everything in the current directory :/
Incidentally, it is not *that* hard to install a DMod.
.tar is not a compression format, just a portable archive one, and the _slightly_ increasing size is due to the additional information (filename, permissions...) that are usually stored in your FAT, just when you compress a .zip file witht the 'store'/'no compression' method -- and that's why you use gzip or bzip2 on the .tar archive to get what WinRAR would call a "solid" archive.
I think WXP only supports .zip files, but I do not have one installed here, so I cannot test.
Self-Extracting zip files are ok because you still can extract the .zip archive from inside it with an extractor - so you can avoid viruses and use it under GNU. However be sure not to use WinZip, but rather a free application (FreeZip, 7-Zip...), since you must get the registered version of WinZip to make a *legal* SFX archive.
Install Maker uses its own format and though you can use Wine to extract it, it is very tedious, so it's better avoid it.
Btw, if people are too dumb to install a DMod, I guess they are too dumb to change the default install directory.
The ending .dmod extension may cause some troubles on some servers/browsers, since they may display it instead of d/l-ing it. It's better create a drag&drop interface (eg improve CustomFE or any front-end), or drag&drop icon on the desktop (that in this case gets the filename using the application's commnad-line parameters).
I also really like the convention in all the .tar.* I saw to contain a carefully named directory containing all the files - it is better when you use the command-line, so you are sure never to extract everything in the current directory :/
This will definately be the case.
Incidentally, it is not *that* hard to install a DMod.
Yeah it is. But you (and me, usually) just don't see it because we're so used to putting up with the BS. For someone to just play a D-Mod, they have to go through several rather obtuse steps. Most people know how to double click on files and install programs, and this would allow more people to play, and those already playing not spend 30 odd seconds installing a D-Mod every single time.
I think WXP only supports .zip files, but I do not have one installed here, so I cannot test.
Yeah, the information I saw implied this as well. I suppose it doesn't really matter... only developers would really want to open the D-Mods to view/extract individual files, so the usability doesn't have to be there for a typical XP user.
The ending .dmod extension may cause some troubles on some servers/browsers, since they may display it instead of d/l-ing it. It's better create a drag&drop interface (eg improve CustomFE or any front-end), or drag&drop icon on the desktop (that in this case gets the filename using the application's commnad-line parameters).
Fileplanet has no problems serving it, with what I assume is an appopriate MIME type, and I tested it in IE and Firebird (Opera just doesn't like Fileplanet), though Firebird had a weird bug that had the default name of my test as e.bz2.dmod.
I tried it with Cycles of Evil if you wanted to try it yourself.
This will definately be the case.
Incidentally, it is not *that* hard to install a DMod.
Yeah it is. But you (and me, usually) just don't see it because we're so used to putting up with the BS. For someone to just play a D-Mod, they have to go through several rather obtuse steps. Most people know how to double click on files and install programs, and this would allow more people to play, and those already playing not spend 30 odd seconds installing a D-Mod every single time.
I think WXP only supports .zip files, but I do not have one installed here, so I cannot test.
Yeah, the information I saw implied this as well. I suppose it doesn't really matter... only developers would really want to open the D-Mods to view/extract individual files, so the usability doesn't have to be there for a typical XP user.
The ending .dmod extension may cause some troubles on some servers/browsers, since they may display it instead of d/l-ing it. It's better create a drag&drop interface (eg improve CustomFE or any front-end), or drag&drop icon on the desktop (that in this case gets the filename using the application's commnad-line parameters).
Fileplanet has no problems serving it, with what I assume is an appopriate MIME type, and I tested it in IE and Firebird (Opera just doesn't like Fileplanet), though Firebird had a weird bug that had the default name of my test as e.bz2.dmod.
I tried it with Cycles of Evil if you wanted to try it yourself.
I'll get back and be more thorough on this issue, but it seems that any browser using the Mozilla Gecko Engine shortens the filename by cutting off the first two letters off the name when using FilePlanet. I should submit this to BugZilla, but I'm too lazy.
Opera just doesn't like Fileplanet
And yet I've used it to download files off there for over a year. *heh* Maybe you're doing something wrong?
And yet I've used it to download files off there for over a year. *heh* Maybe you're doing something wrong?
OK, right; i think I am just a bit upset when people cannot do things I find very simple
You may want to use ".tar.bz2" or the shorter ".tbz" in coe".bz2".dmod, so the extension still shows all the information about the compression method.
I could download the file perfectly from FilePlanet; however, I am not sure it will be ok with other servers: I tried on my personal Apache with default configuration, and it uses a text/plain MIME type for unknown file types, so the archive is displayed in the browser, whichever it is.
Well, one could strip the ".dmod" part if such a problem occurs.
Everything else seems very fine
You may want to use ".tar.bz2" or the shorter ".tbz" in coe".bz2".dmod, so the extension still shows all the information about the compression method.
I could download the file perfectly from FilePlanet; however, I am not sure it will be ok with other servers: I tried on my personal Apache with default configuration, and it uses a text/plain MIME type for unknown file types, so the archive is displayed in the browser, whichever it is.
Well, one could strip the ".dmod" part if such a problem occurs.
Everything else seems very fine
The results are (sorta) in, and 7-zip is DEFINITELY out. I have used:
* PowerArchiver 8.60.02
* 7-zip 3.11
* GNU tar 1.13
* GNU gzip 1.3.5
* GNU bzip 1.0.2
-- The D-MODs Legend of Tenjin (LOT) and Stone of Balance (SOB) were used as benchmarks. The original sizes were -
LOT: 81,722,682 bytes
SOB: 64,408,690 bytes
In compression:
PowerArchiver:
LOT-
Format | Time | New Compressed Size
.tar - 5.37s | 83,148,800
.tar.gz - 12.84s | 7,192,127
.tar.bz2 - 16.75s | 6,217,091
.7z - N/A
SOB-
Format | Time | New Compressed Size
.tar - 7.35s | 65,853,440
.tar.gz - 12.73s | 9,450,568
.tar.bz2 - 17.04s | 8,655,353
.7z - N/A
7-zip:
LOT-
Format | Time | New Compressed Size
.tar - 4.18s | 83,167,744
.gz - N/A
.bz2 - N/A
.7z - 92.23s | 5,448,617
SOB-
Format | Time | New Compressed Size
.tar - 5.93s | 65,883,136
.gz - N/A
.bz2 - N/A
.7z - 91.98s | 7,223,708
GNU tar
LOT - 4.69s - 83,169,280
SOB - 2.98s - 65,884,160
GNU gzip (WITH tarring previously done)
LOT - 11.26s - 7,635,323
SOB - 13.31s - 9,254,176
GNU bzip2 (WITH tarring previously done)
LOT - 15.95s - 6,317,125
SOB - 21.74s - 8,623,692
And the decompression times:
LOT -
.tar - 17.92s
.gz - 17.97s
.bz2 - 16.87s
SOB -
.tar 17.42s
.gz - 18.89s
.bz2 - 17.38s
Okay, so, they're probably a little off. Interestingly enough, the first time I tried compressing them, it took around 20 seconds. The second, third, fourth, and fifth times took about 5 seconds. That kinda lowered the average. Would there be any caching?
--
OK, right; i think I am just a bit upset when people cannot do things I find very simple
Uh, who are you talking to?
* PowerArchiver 8.60.02
* 7-zip 3.11
* GNU tar 1.13
* GNU gzip 1.3.5
* GNU bzip 1.0.2
-- The D-MODs Legend of Tenjin (LOT) and Stone of Balance (SOB) were used as benchmarks. The original sizes were -
LOT: 81,722,682 bytes
SOB: 64,408,690 bytes
In compression:
PowerArchiver:
LOT-
Format | Time | New Compressed Size
.tar - 5.37s | 83,148,800
.tar.gz - 12.84s | 7,192,127
.tar.bz2 - 16.75s | 6,217,091
.7z - N/A
SOB-
Format | Time | New Compressed Size
.tar - 7.35s | 65,853,440
.tar.gz - 12.73s | 9,450,568
.tar.bz2 - 17.04s | 8,655,353
.7z - N/A
7-zip:
LOT-
Format | Time | New Compressed Size
.tar - 4.18s | 83,167,744
.gz - N/A
.bz2 - N/A
.7z - 92.23s | 5,448,617
SOB-
Format | Time | New Compressed Size
.tar - 5.93s | 65,883,136
.gz - N/A
.bz2 - N/A
.7z - 91.98s | 7,223,708
GNU tar
LOT - 4.69s - 83,169,280
SOB - 2.98s - 65,884,160
GNU gzip (WITH tarring previously done)
LOT - 11.26s - 7,635,323
SOB - 13.31s - 9,254,176
GNU bzip2 (WITH tarring previously done)
LOT - 15.95s - 6,317,125
SOB - 21.74s - 8,623,692
And the decompression times:
LOT -
.tar - 17.92s
.gz - 17.97s
.bz2 - 16.87s
SOB -
.tar 17.42s
.gz - 18.89s
.bz2 - 17.38s
Okay, so, they're probably a little off. Interestingly enough, the first time I tried compressing them, it took around 20 seconds. The second, third, fourth, and fifth times took about 5 seconds. That kinda lowered the average. Would there be any caching?
--
OK, right; i think I am just a bit upset when people cannot do things I find very simple
Uh, who are you talking to?
OK, right; i think I am just a bit upset when people cannot do things I find very simple
Uh, who are you talking to?
Me, about the 'installing D-Mods is easy' thing.
Uh, who are you talking to?
Me, about the 'installing D-Mods is easy' thing.
I just have a quick question,(not sure how I got in this forum), but aren't some of your compressed files larger than the uncompressed originial? Win XP has winzip & it works great. Unzipping is easy as pie, the other one, ClickTeam, is a pain in the a** to open.
Yay. Bzip2 support has been added to Windemere. It uses polymorphism for my sake of sanity (or what little is left) , which means any part of dink can easily access that .dmod file using as little memory as possible. Next comes tar...
, which means any part of dink can easily access that .dmod file using as little memory as possible. Next comes tar...
Amazing! I read this whole thing and understood about 1/32. It was almost like trying to read an alien language. I am astounded at all the talent on this board. I believe you guys could create almost anything when you put your heads together and some of you (wait for it....hi) will be the B.(i)G. of the future. WOW
