| |
Shadow
Account closed
Registered: Apr 2002
Posts: 355 |
Load first decrunch later or stream decrunch?
Work on my first attempt on an IRQ-loading demo continues. I am using Dreamload, and everything is working great.
However, some parts tend to be rather big and eat both diskspace and load time.
Obvious solution would of course be to compress the data.
As I see it, there should be two options.
1. Load entire compressed part, then decompress.
2. Load byte-by-byte and stream-decompress
At the moment I am leaning towards solution 1. To get this working, the unpacking must allow for overlapping between packed and unpacked data, since I don't have space for both obviously. But I guess a smart decompressor works 'in reverse' so to speak, so overlapping should not be a problem as long as unpacked data is larger than packed...
I have looked at Exomizer, and it seems like it does things that way, and the decompressor code is fairly compact, so it could be a way to go.
Option 2 I have not looked into as much. Dreamload does support a callback on byte-for-byte basis, so it should be possible I guess.
So, I ask all veterans in this field - how is it usually done? Any tips and general good ideas? |
|
... 59 posts hidden. Click here to view all posts....
|
| |
HCL
Registered: Feb 2003
Posts: 717 |
@doynax: Oh my, stop disassembling :). it's here: ByteBoozer V1.0. There is even a loader with integrated ByteBoozer decruncher, check out the testprog.. All in TurboAssembler phormat though, haven't exported it to PC-textfile. |
| |
doynax
Account closed
Registered: Oct 2004
Posts: 212 |
Quote: @doynax: Oh my, stop disassembling :). it's here: ByteBoozer V1.0. There is even a loader with integrated ByteBoozer decruncher, check out the testprog.. All in TurboAssembler phormat though, haven't exported it to PC-textfile.
Gah! How did I manage to miss that one?
Now then, lets see if I can add a sliding window to get true streaming decompression and perhaps see if there's anything to gain by optimizing for speed rather than size.
As usual I'm feeling an irresistible urge to reimplement my own new and slightly crappier version of the whole thing. Goddammit this project will never get finished.. |
| |
AlexC
Registered: Jan 2008
Posts: 293 |
Reading this very interesting topic I started to think about using additional ($2000-$BFFF) drive memory. While I'm aware that extensions like RAMBoard aren't that common let's face it: at one point we will be forced to forget about original drives and MMC64 and 1541Ultimate are probably just the beginning of 1541 evolution and Vice support additional memory without any problems. Anyway I was wondering if somebody did some some testing in decrunching in drive and than sending code to c64. Haven't experimented with transmission yet so I don't have any idea about potential bottlenecks. |
| |
MagerValp
Registered: Dec 2001
Posts: 1059 |
I'm working on ULoad Mini, optimized to use a little ram as possible in the C64. It does LZMV decrunching using a 256-byte ring buffer in drive ram, and sends over an uncompressed stream. The performance hit is pretty severe, but the loader fits in a fraction of the space on the C64 side (currently 138 bytes for full 16 char filenames, less if you go for 2 char).
|
| |
Oswald
Registered: Apr 2002
Posts: 5025 |
1541u or mmc should become kind of a standard setup, and even supported in emulators to this become a reality. I don think it would, since soon 30 yrs the standard is c64+drive. |
| |
AlexC
Registered: Jan 2008
Posts: 293 |
Quote: I'm working on ULoad Mini, optimized to use a little ram as possible in the C64. It does LZMV decrunching using a 256-byte ring buffer in drive ram, and sends over an uncompressed stream. The performance hit is pretty severe, but the loader fits in a fraction of the space on the C64 side (currently 138 bytes for full 16 char filenames, less if you go for 2 char).
Very interesting. Is there any chance to take a loot at it? This make it perfect solution for releases that occupy almost all free memory for example. I can image that the user can wait a bit longer during initial loading to be free from additional loading during gameplay for example. Sometimes even switching out IO space at $D000 for relocator for example is not enought to fit everything into c64 RAM. |
| |
AlexC
Registered: Jan 2008
Posts: 293 |
Quote: 1541u or mmc should become kind of a standard setup, and even supported in emulators to this become a reality. I don think it would, since soon 30 yrs the standard is c64+drive.
At one point they will all break due to mechanical reasons for example. Secondly I prefer - I'm guess I'm not the only one - loading files from SD card for example. |
| |
HCL
Registered: Feb 2003
Posts: 717 |
Hmm.. and sooner or later all those 6510-chips gotta break up also, so why don't we better code PC-demos instead?
No kidding.. I'm not saying we should not use 1541U, but the day we stop supporting the original 1541 drive we have probably taken a step in the wrong way. |
| |
MagerValp
Registered: Dec 2001
Posts: 1059 |
Quote: Very interesting. Is there any chance to take a loot at it? This make it perfect solution for releases that occupy almost all free memory for example. I can image that the user can wait a bit longer during initial loading to be free from additional loading during gameplay for example. Sometimes even switching out IO space at $D000 for relocator for example is not enought to fit everything into c64 RAM.
The 1-bit protocol I was going to use (which is basically the same as Covert Bitops et al) doesn't work in this scenario - I need a protocol that does generic getbyte/sendbyte. Currently the loader just hangs on startup, so it's of little use to anyone right now.
|
| |
AlexC
Registered: Jan 2008
Posts: 293 |
Quote: The 1-bit protocol I was going to use (which is basically the same as Covert Bitops et al) doesn't work in this scenario - I need a protocol that does generic getbyte/sendbyte. Currently the loader just hangs on startup, so it's of little use to anyone right now.
Thanks for the feedback. BTW: have you considered burst option in c128/1571? |
| Previous - 1 | 2 | 3 | 4 | 5 | 6 | 7 - Next |
