Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #3909 > unrolled thread

i need help in compression algorithm identification

Back to article view | Back to comp.compression

Contents

  i need help in compression algorithm identification theuswra@gmail.com - 2020-06-29 20:51 -0700
    Re: i need help in compression algorithm identification Helm <nospam@please.invalid> - 2020-06-30 09:35 -0400
      Re: i need help in compression algorithm identification news@zzo38computer.org.invalid - 2020-06-30 13:14 -0700
      Re: i need help in compression algorithm identification williamvazmolina1998@gmail.com - 2020-06-30 22:12 -0700
        Re: i need help in compression algorithm identification theuswra@gmail.com - 2020-07-01 00:19 -0700
          Re: i need help in compression algorithm identification Helm <nospam@please.innvalid> - 2020-07-01 07:05 -0400

#3909 — i need help in compression algorithm identification

okay so, i'm modding a game which has compressed files, as far i could see, it has apparently 2 "compression patterns", files starting with "00 05" and "00 07". I was able to get some samples (compressed files) and the respective decompressed ones, (through the in-game memory the decompressed ones are accessible). i'm asking for help because i have no experience in that matter, and i hope someone could help me.

samples: https://drive.google.com/file/d/1slzoYxqr3Egg3Z7GCKlayTNwCR7OZ2Yg/view?usp=sharing

[toc] | [next] | [standalone]

#3910

On 2020-06-30 03:51:13 +0000, theuswra@gmail.com said:

> okay so, i'm modding a game which has compressed files, as far i could 
> see, it has apparently 2 "compression patterns", files starting with 
> "00 05" and "00 07". I was able to get some samples (compressed files) 
> and the respective decompressed ones, (through the in-game memory the 
> decompressed ones are accessible). i'm asking for help because i have 
> no experience in that matter, and i hope someone could help me.
> 
> samples: 
> https://drive.google.com/file/d/1slzoYxqr3Egg3Z7GCKlayTNwCR7OZ2Yg/view?usp=sharing 
> 

First off; what game is this? If it runs on a certain engine that can 
give us clues. I know of certain engines  that have their compression 
scheme well documented. Otherwise you will have to reverse engineer the 
decompress function to find out how it does it - probably some 
DEFLATE/LZW stuff . With the decompressor code (which is obviously 
present), and knowledege of the compression method used/compression 
methods in general, you should be able to create a (probably very 
inefficent) compressor.

Luckily for you, most game developers are not compression experts and 
looking at the compression ratio its probably something very simple.

I assume you know much about reverse engineering since you are modding 
a game, but you should take some time to read up on Huffman encoding 
etc. so you understand the code.

Second of all; when you post on Google Groups, it places your IP 
address and your email in the headers. Consider downloading a 
newsreader and using the free newsserver aioe.org (no registration 
required) to maintain your privacy and prevent your IP from leaking 
out. In addition, many people on Usenet have Google Groups killfiled, 
so some wont see this post until someone who doesn't use google groups 
(like me) reply to it. The only time I use google groups is for reading 
Usenet on the go since iOS has no good newsreader (even paid newstap is 
crap) or when I need to send someone a link to a Usenet post. I will 
always post using my newsreader instead.

[toc] | [prev] | [next] | [standalone]

#3911

Helm <nospam@please.invalid> wrote:
> Luckily for you, most game developers are not compression experts and 
> looking at the compression ratio its probably something very simple.

What I know is that I tended to make my own compression for the game,
which is sometimes such that unsolvable levels sometimes cannot be
represented in the compressed format at all. Note, however, that in some
games, uncompressing random data will not necessarily result in a solvable
level; in other games, it does. (Do any other games do this? How common is
it? I am curious to know. I think I read that at least Gruniozerca 3 does
something similar; the compression algorithm relies on the solution, so it
is also easily possible to determine the solution from the compressed
level data, too.) (I remember actually that in one computer game I wrote,
uncompressing random data (if it is sufficiently long) always does result
in a solvable level, and that is actually how the "random levels" mode of
the game works; it uncompresses random data (based on a random number seed
entered by the user, so that you can easily find and access the same level
later). The solution is also easily derived from uncompressing it, so it
also has solutions, too.)

> [...]                  The only time I use google groups is for reading 
> Usenet on the go since iOS has no good newsreader (even paid newstap is 
> crap) or when I need to send someone a link to a Usenet post. I will 
> always post using my newsreader instead.

My suggestion is when using Google Groups to link to a Usenet post, to
always use the URL based on the message ID. Not everyone uses Google
Groups, and using the URL based on the message ID makes it helpful for
those who do not use Google Groups. (I do not use Google Groups myself.)

(I can also perhaps suggest trying to write a better newsreader for iOS,
if a suitable one does not already exist. Maybe it is possible to use SSH
to access it on another computer, though?)

-- 
This signature intentionally left blank.
(But if it has these words, then actually it isn't blank, isn't it?)

[toc] | [prev] | [next] | [standalone]

#3912

The game is: "Naruto Uzumaki Chronicles 2" (PS2)

[toc] | [prev] | [next] | [standalone]

#3913

The game is Naruto Uzumaki Chronicles 2 (PS2) by Racjin, btw they use the same compression format for all their games (the games with compressed files, ofc.) and i run it with pcsx2.

Also i dont think that recompress the files is necessary tbh, i just need to mass decompress them without having to do it manually, file by file through memory.

Sorry for any english mistakes, i'm not a native english speaker.

[toc] | [prev] | [next] | [standalone]

#3914

On 2020-07-01 07:19:55 +0000, theuswra@gmail.com said:

> The game is Naruto Uzumaki Chronicles 2 (PS2) by Racjin, btw they use 
> the same compression format for all their games (the games with 
> compressed files, ofc.) and i run it with pcsx2.
> 
> Also i dont think that recompress the files is necessary tbh, i just 
> need to mass decompress them without having to do it manually, file by 
> file through memory.
> 
> Sorry for any english mistakes, i'm not a native english speaker.

I had a feeling by looking in the files. MIPS is not my cup of tea. 
Well if you just need to mass decompress them then you still need to 
figure out the algorithm..... I dont think the ps2 makes use of ASLR so 
it should be fairly straightforward to find out where it will load the 
decompressed file, trace memory and go from there (dynamic analysis)... 
without any function names, it gets harder. Maybe look for all syscalls 
to open a file and see if you can get the decompression method.


I hear there was a PC version, if they were using the same engine then 
probably the same compression method was in effect, I don't know if 
Windows or PS2 reversing is easier for you.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.compression

csiph-web