What's new

Melee Arcive Viewer (MAV) in development project.

Tcll

Complexity == Fun >:3
well... due to jahra!n and the other's giving up on their projects...

I plan to start one of my own.

Melee Archive Viewer or MAV basically does what you'd expect any archive viewer to do.
opening the archive, viewing the data and extracting it, or editing and compiling it...
I got the idea from BrawlBox 0.63b
which does what I want my program to do.

my only problem is...
I'm still an amature Python programmer, and I need some help.
 
Last edited:
OP
Tcll

Tcll

Complexity == Fun >:3
my first thing I'd like to know is well...
how exactly do I build a module that can recognise the DAT format structure??

I mean...
What exactly am I looking for??
 
Last edited:

revel8n

New member
hello,
guess i'll chime in here. i have plans to get back to this format as well as the Metroid Prime and Brawl formats again soon.

You want to look for patterns within the binary data of the files. Compare multiple files and see if there are any consistent similarities. It is often best to start with small files of the same type, and if necessary comparing against a medium to large file to see if any of those assumed consistencies can be confirmed. Header information will usually be contained at the beginning of the file so you can often make some good assumptions from the first 10s of bytes or so.

For formats that are completely unknown you will have to start making assumptions about values within the files, then take steps to attempt to verify or deny the viability of those assumptions. Scan the file for things you can readily prove....counts of data entries, consistent placement of data patterns, etc. i.e., If you find a section with a list of string data, attempt to count the number of strings manually, then check for that count somewhere in the determined header information. The same goes for data patterns.

Since the file needs to be read by the game there is more than likely going to be some way for it to determine where data is located within the file. The most common way of doing this is by explicitly listing an offset somewhere withing the header or other data structures contained within the file. This is usually the biggest back and forth area when it comes to attempting to interpret the data, because given the nature of file formats you are likely to have to decide whether a file represents a count of some sort or an offset within the file. As far as offsets are concerned you have at least on piece of information right off the bat that help only slightly in validating offsets, that's the file size. If a value is larger then the possible file size, it is highly unlikely it is a valid offset (if its reasonably close, and you have suspicions the file is compressed, it could be the uncompressed size, but that's for another topic, heh). Another common thing that can help identify file offsets is the tendency for data to be stored aligned to the size of various computer data type sizes and quite often power of 2 boundaries. This is not always the case, but it can at least initially rule out a lot of values that do not fall within even numbers. 4, 8, and 16 byte alignments are quite common.

Since the files should usually follow some sort of structured layout, once you have assumed or determined a file offset, time to see whats there. And since multiple file should follow the same or very similar structure, you should be able to find the same or similarly inclined offset in another file of the same type. Then its on to comparing the data at those locations, and the cycle repeats, with things being verified, disproved, leading to new assumptions being made, thus more verifications, and so on. Not all formats include every count or offset, and it can often be inferred from the counts that are known, and the sizes of the data itself, will have to experiment in some cases to determine the difference.

Determining the intent of data values you find after you have exhausted offset and count validations, can be a bit tricky. Any hex editor worth using should allow for you to visually see how data at a particular offset is/would be interpreted when assumed to be various data types (bytes, shorts, longs, floats, doubles, etc.), though i guess if you want you could always work it out by hand or in your head if you mind works that way, heh. Having an idea of the type of data you are looking for and how it may or should be structured can be very helpful here. in particular for this case, the gamecube/wii has a very particularly way of receiving its data for rendering, and may be something you want to look into should you be inclined to look into more games on these systems.

Sorry to have been so long winded in getting to this point. Now as to the Melee formats in question. The *.dat file start off pretty similarly in terms of structure. The header is 0x20 (32) bytes long, and all file offset are relative to the end of the header (thus they all need 32-bytes added to them). Determining this comes from file comparisons or even better if you can find any related loading code in the game executable. Also of note is that all the data values are in Big Endian, thus you should be able to read them fairly easily from within a hex editor, but unless you are on a computer using the same format, the data will need to be byte swapped before it can be interpreted correctly.

If you look at the first 4 bytes of the files, and start off assuming it is offset related, you will notice from your first round of of offset validation checks that it is exactly the same as the file size. Cannot really have data located at the very end of the file, so it is probably not an offset, but since it is so close to (exactly) the actual size of the file, it is probably not a count of some sort either, but the actual file size stored explicitly in the header data. The next 4 byte value after that you might notice is smaller, yet quite close to the previous file size value. You could then attempt to go to that value offset in the file and see whats there, but you have to make sure to take into account the fact that all offsets are relative to the end of the header, so adding 32 to the value will probably be a bit more useful to finding a proper location to investigate. You need to also make sure to take this into account when investigating any other assumed offsets you want to look into, as well as if you want to check whether or not some data you have found in the file has its offset listed somewhere else.

Like i said, i have not gotten back to finishing this format yet, so there are still a number of things i need to look into. i may have a bit more information, i will see what i can dig up.

Hope that helps.
 
OP
Tcll

Tcll

Complexity == Fun >:3
It did help out
thanx alot

have you seen what I've found out so far??
it's about what the outside files are...
but it's not quite finished yet..., still got a few more files to go...
http://smashbrosfiles.blogspot.com/p/technical-file-information-melee.htmlhttp://ssbmfiles.blogspot.com/
you probably already know how I found that out...
if not, it is listed.
________________________________________________________________
... and INSERTING edited stuff BACK to the game?

that would be compling :)
 
Last edited:
OP
Tcll

Tcll

Complexity == Fun >:3
I've been assuming the images might be TGA...
but now I'm thinking they might be TPL files like Brawl's data...

what do you think??
 

Milun

New member
The images are .tga, converted into .tpl when placed in the game. If you'd like to know more on texture hacks, visit here.
 

Myuutsuu

forever.
Awesome, a MeleeBox in the works, haha. That'd be really sweet.

As Milun already pointed out, Melee's texture files are in .TGA format, VERY similarly to Brawl... The two games share quite a bit in common on those grounds (given Brawl was supposed to be Melee 2.0 + online). And, in the topic Milun supplied, there's a boatload of information available that you will hopefully find of some use... especially in the pages/posts after the first 20 or so. :)
 
OP
Tcll

Tcll

Complexity == Fun >:3
I kinda wan't to build a program that works like Paint, but only with TPL files...

don't really want to go through the work of building 2 progs though...


____________

Always had this Q going through my head...

Can you see my sig here??
 
OP
Tcll

Tcll

Complexity == Fun >:3
Before I actually start coding this however...
I'm going to code a Blender Import script (fully)

meaning you should be able to import the entire char into Blender...
IPO Animations and all.

even though Blender can do the same exact things as MAV...
MAV is just a more user friendly way to view the char + export and import data.
this means I'm building it for everyone... (for the slower people (like me...))
 
OP
Tcll

Tcll

Complexity == Fun >:3
hey Milun :O

I thought of a better way to explain the face hex...
you only highlighted the first parts... not the whole part
here is your code:
98 00 7B ;descriptor
03 15 03 15 01 A6 ;V-Point X face 1
03 16 03 16 01 A7 ;V-Point Y face 1
03 17 03 17 01 A8 ;V-Point Z face 1
03 16 03 16 01 A7 ;V-Point X face 2
03 10 03 10 01 A1 ;V-Point Y face 2
03 16 03 16 01 A7 ;(V-Point X face 3)
03 13 03 13 01 A4 ;V-Point Z face 2
03 15 03 15 01 A6 ;and so on...
03 12 03 12 01 A3
03 11 03 11 01 A2
03 12 03 12 01 A3
00 A8 00 A8 01 A6
03 13 03 13 01 A4
00 A9 00 A9 01 A7
00 72 00 72 01 A1
00 A9 00 A9 01 A7
00 E1 00 E1 01 A8
00 A8 00 A8 01 A6
00 DE 00 DE 01 A5
03 11 03 11 01 A2
00 DE 00 DE 01 A5
00 DF 00 DF 01 61
00 E1 00 E1 01 A8
00 E0 00 E0 01 62
00 4B 00 4B 01 5F
00 E0 00 E0 01 62
01 19 01 19 01 63
00 DF 00 DF 01 61
01 16 01 16 01 60
03 11 03 11 01 A2
01 16 01 16 01 60
01 17 01 17 01 2A
01 19 01 19 01 63
01 18 01 18 01 2B
00 36 00 36 01 28
01 18 01 18 01 2B
01 4F 01 4F 01 2C
01 17 01 17 01 2A
01 4C 01 4C 01 29
03 11 03 ...unfinished
when you put the data differently... you get different ideas...
notice how the 5th column is nothing but 01...

and don't ignor that duplicate data...
it basically means 2 face verts (in the same spot) that arn't connected.

dont quote me on this however...
this is pretty much just a really good guess...

All data means something. (a good rule to go by)
 
Last edited:
OP
Tcll

Tcll

Complexity == Fun >:3
To Milun:
I might be able to help you out with the UV's...

so let's see...
I know they only have an x,y axis
and I know that they're farmatted the same as verts

but what I don't know is how they're linked to the mesh :(

any ideas??
 

Milun

New member
Well I'm back, and I have to say how extremely impressed I am with your edge data discovery. I was tearing my hair out trying to figure it out. Oh, and I'm sorry, but I can't help you with the textures, mainly because of how pre-occupied I was with finding the edges, but I'd wager that the last value in the above column has something to do with it, but I'd have to research it to be sure.
 
OP
Tcll

Tcll

Complexity == Fun >:3
lol
that's ok...
I'll try to find it when I can...

btw: your vert data helped me out alot.
here's a first run sceenshot of my test prog in blender

click on the image for a larger shot.
see if you can guess who it is before looking at any data or clicking on the image. :D

but yea... you can see my prob...
it seems alot of the Z axis likes to go to 0 before being exported
and btw... this isn't the original import either

here is my 1st test obj
Download
 
Last edited:

Milun

New member
Well, from your example I downloaded, I can't exactly tell what's the problem with the Z values, but I see you forgot the:

"If the number is smaller or equal to 127, then it stays the same.
If the number is larger than 127, then 256 must be subtracted."

That would explain why it was divided into four sections, but I still don't know what's wrong with the Z, my advice is to check your code on a .ty file (trophy) such as the TyEgg.dat.

Trophies are easier:



Edit: Hmmm... there's also this problem:



And incidentally, this is the best I've ever accomplished (shown in steps):


Every face hex is in the leftmost two columns (it's for the Mario trophy btw), but notice these ones skip less hex than the one you showed.
 
Last edited:
OP
Tcll

Tcll

Complexity == Fun >:3
actually...
I didn't forget that.

the original numbers are over 400 so I did subtract the 256 value from them...
but was I supposed to go back and redo it??
if the numbers are still bigger than 127...

as of your problem...
it's got me stumped... o_O

however I'm not there yet
I'm still on the verts... :p

EDIT:
I know what's wrong with mine however...
X is rotated by 90 so that turns Y into Z
so that means the prob lies within the Y convertor

My program converts the raw data into hex before converting that into a decimal value.
this means my prob is the raw to hex conversion on the Y values
or tmp3 and tmp4
(the vars that hold those values)

this is my final format
vert = "v "+tmp1+"."+tmp2+" "+tmp3+"."+tmp4+" "+tmp5+"."+tmp6
this is how I have it formatted to output the obj decimals
 
Last edited:

revel8n

New member
hey Milun :O

I thought of a better way to explain the face hex...
you only highlighted the first parts... not the whole part
here is your code:
98 00 7B ;descriptor
03 15 03 15 01 A6 ;V-Point X face 1
03 16 03 16 01 A7 ;V-Point Y face 1

First part should be interpreted like this:

98 00 7B ; primitive flags, index count

Code:
primitiveFlags & 0xF8

0xB8 // points
0xA8 // lines
0xB0 // line strip
0x90 // triangles
0x98 // triangle strip
0xA0 // triangle fan
0x80 // quads

The index data start with a byte value that determines the primitive type to be rendered, then a 2-byte value that determines the number of index groups that follow. Unfortunately this is where things can get trick, because not all models will have normals, and untextured models may not have uvs, or if a model ever has multiple textures there could be multiple uv indices. As such there may not always be an index group size of 3 like is being assumed here (vertex, normal, uv index values per indexed vertex in this case i am assuming from what has been shown). Also for models that have less than 256 values to index, it is not always guaranteed that index values will even be 2 bytes. Just something to be aware of if you see some files that don't seem to match the pattern you are adhering to now.

03 15 03 15 01 A6 ;Vertex 1 (vertex index, normal index, uv index) *assuming i have the order correct

After count number of groups, the sequence may repeat with another primitive type byte and 16-bit index count and so on. May help you to look up some info on how the gamecube/wii send data for rendering to the graphics card.

Hope that helps.
 
OP
Tcll

Tcll

Complexity == Fun >:3
KK... o_O
as I said "Don't quote me on that" XD
I thought I might have been off...
but yea..., I'm going to be looking back on this.
thanks

I do wish Jahra!n was here... :(
------------------------------------------------------------------------------------------

btw Milun...

I tried out your stratagy of putting the hex data from HxD into a txt.

it worked out better than my stratagy...
but I couldn't help but notice how none of the verts were in the right place

the model was a complete mess
they were off by values of at least 0.1 to 0.3
 
Last edited:

revel8n

New member
Oh, also, make sure you are treating values as signed when necessary.

To take an example from Milun's page:
(F5 50 becomes 62800) as unsigned is very different when compared to
(F5 50 becomes -2736) when treated as a signed value.
 
Last edited:
OP
Tcll

Tcll

Complexity == Fun >:3
Oh, also, make sure you are treating values as signed when necessary.

To take an example from Milun's page:
(F5 50 becomes 62800) as unsigned is very different when compared to
(F5 50 becomes -2736) when treated as a signed value.

ok...
being an amature py programmer <XD
umm... I don,t quite get the differance between them... :(

or should I say, I'm not quite sure how to do that...
 

Top