Ebootr isnt dead!

[indrora]

Kay y'all, here's my current update.

I havent been able to put as much effort into Ebootr as I'd like! There's still several major chunks of code I really *really* need to optimize, but Here's What I've Done So Far:

• SFO parsing!
• SFO dumping in main window!
• SFO Writing!
• A really hacked together SFO editor!
• Some minor things that you proabably wont see....

The SFO parsing reads strings a little funky, and it *will* include multiple NULLs if you use my library! However, on the flipside, it packs things *tighter* than PBP Packer! QuakePSP's Param.SFO was 904 bytes when I started, was only 122 when I finished.

I really must thank the guy who came up with Another PSP Documentation Page (Its got a *really* Nice description of how the SFO (PSF) format works.

I'm making SFO objects look like this for everyone who cares:

SFOFile sfo = new SFOFile(); // new SFOFile
sfo.Add(SFOFile.GetKeyPair("TITLE", "My Kick-ass PSP app")); // Add a string  item to it.
sfo.Add(SFOFile.GetKeyPair("BOOTABLE", 1));  // Add an integer to it.
sfo.Add(SFOFile.GetKeyPair("DRIVER_PATH", System.Text.Encoding.UTF8.GetBytes("SYSDIR"))); // add binary data to it.

{ SNIP }
SFOFile sfo = new SFOFile(System.IO.File.ReadAllBytes("param.sfo")); // new SFO file from byte[] stream.
if(sfo.ContainsKey("TITLE")) // Does it contain the "TITLE" key? (NOTE! this is MOST DEFINITELY case sensitive!)
{
 MessageBox.Show(sfo["TITLE"].ToString()); // show it!
}

[slasher101]

Haha You coding in C#

You know you can't code in C# for PSP :/

[seanpaul223]

Maybe, but, as far as I know, Ebootr is some sort replacement for PBP Unpacker, thus for PC and not for the PSP.
Am I wrong ?

[indrora]

Its a PC-Side app to replace PBP Unpacker because I dont like the style.

One thing I may do is include a command line version of the tools to make life so much easier.

There is a version of the CLR for the PSP, but I havent tried playing around with it.

[slasher101]

I can help you if you need help, Im Mast3r C# Codaaaaar

[indrora]

Maybe.
I'm working on getting pseudo-SVN from my webhost....

Currently, its ~2kloc
... here's just for reading SFO files!

Spoiler for Code...:

namespace Ebootr
{
    // Based off an ICollection because they auto-sort alphabetically.
    public class SFOFile : IDictionary<string, SFOFile.SFOEntry>
    {

        public static byte[] ParseHexString(string input, char[] sep)
        {
            byte[] q = new byte[input.Length]; // For paranoia :)
            string[] in_broken = input.Split(sep);
            int idx = 0;
            foreach (string s in in_broken)
            {
                string qx = s;
                qx = s.Replace("0x", "");
                qx = qx.Replace(" ", ""); // strip spaces, because we care.
                byte c;
                if (!Byte.TryParse(qx, out c))
                {
                    throw new ArgumentException("Value cannot parse to byte.", "input:" + idx + "=" + qx);
                }
                q[idx] = c;
                idx++;
            }
            return q;
        }

        public static KeyValuePair<string, SFOEntry> GetPair(string name, string value)
        {

            return new KeyValuePair<string, SFOEntry>(name, new SFOEntry( value ) );

        }

        public static KeyValuePair<string, SFOEntry> GetPair(string name, UInt32 value)
        {
            return new KeyValuePair<string, SFOEntry>(name, new SFOEntry( value ) );

        }
        public static KeyValuePair<string, SFOEntry> GetPair(string name, byte[] value)
        {
            return new KeyValuePair<string, SFOEntry>(name, new SFOEntry( value ) );

        }



        /// <summary>
        /// SFO Entry (Entry into the SFO Table.
        /// </summary>
        public struct SFOEntry
        {
            public SFOEntryType type;
            public byte[] data;

            public SFOEntry(SFOEntryType t, byte[] d)
            {
                type = t;
                data = d;
            }

            public SFOEntry(string value)
            {
                type = SFOEntryType.txt;
                data = System.Text.Encoding.UTF8.GetBytes(value);
            }

            public SFOEntry(UInt32 value)
            {
                type = SFOEntryType.num;
                data = BitConverter.GetBytes((UInt32)value);
            }
            public SFOEntry(byte[] value)
            {
                type = SFOEntryType.bin;
                data = value;
            }




            public override string ToString()
            {
                switch (this.type)
                {

                    case SFOEntryType.txt:
                        // Apparently, we're going to have to start using this. Its sad, but true, that there are almost always NULLs added. 
                        return System.Text.Encoding.UTF8.GetString(data).Replace("\0", "");

                    case SFOEntryType.num:
                        return "" + BitConverter.ToUInt32(data, 0);

                    case SFOEntryType.bin:
                    default:
                        string v = "";
                        foreach (byte b in data)
                        {
                            v += String.Format("0x{0:2X}", b);
                        }
                        return v;
                }
            }
        }

        /// <summary>
        /// The type of entry an SFOEntry is.
        /// </summary>
        public enum SFOEntryType : byte
        {
            bin = 0,
            txt = 2,
            num = 4
        }
        // the filetype. NOTE!!! this isnt really useful. 
        public enum SFOFileType
        {
            SaveGame,
            MSGame,
            UMDGame

        }
        /// <summary>
        /// UNUSED
        /// </summary>
        [StructLayout(LayoutKind.Sequential, Pack = 4)]
        struct SfoHeader
        {
            UInt32 magic;
            UInt32 version;
            UInt32 key_table_offset;
            UInt32 value_table_offset;
            UInt32 number_values;
        }

        /*
         * Here's how the Param.SFO file is laid out.
         * 
         * HEADER:
         * S    E   L   NOTES
         * 0    3	4	0 "PSF" A file type identification cookie. A zero byte is followed by the three uppercase ASCII characters "PSF".
         * 4	7	4	1 1 0 0 This might be some kind of indication of the PSF version. Currently it's always two 1 bytes followed by two 0 bytes.
         * 8	11	4	ul32 Offset from the start of the file to the start of the key table (in bytes)
         * 12	15	4	ul32 Offset from the start of the file to the start of the value table (in bytes)
         * 16	19	4	ul32 Number of key/value pairs in the index
         */



        /// <summary>
        /// Constructor for an SFOFile.
        /// </summary>
        public SFOFile()
        {
            Dict = new SortedDictionary<string, SFOEntry>();

        }

        /// <summary>
        /// Constructor for an SFOFile that takes in an array of bytes to read from.
        /// </summary>
        /// <param name="qq">Binary data for the SFO file.</param>
        public SFOFile(byte[] qq)
        {
            Dict = new SortedDictionary<string, SFOEntry>();

            // first, we're going to make sure that we know the first offset

            // offset of key table
            uint key_offset = BitConverter.ToUInt32(qq, 8);
            // offset of values table
            uint val_offset = BitConverter.ToUInt32(qq, 12);

            // number of entries.
            uint num_entries = BitConverter.ToUInt32(qq, 16);

            for (int idx = 20; idx < (16 * num_entries) + 20; idx += 16)
            {

                // our current item is 16 bytes long.
                // Copy the current idx to idx+16 into our current item buffer.

                byte[] current_item = new byte[16];
                Array.Copy(qq, idx, current_item, 0, 16);

                // information we need to know about our current item.
                UInt16 keyname_offset;
                byte data_type, unknown_field;
                UInt32 value_size;
                UInt32 value_size_padded;
                UInt32 data_offset;

                // get the information using BitConverter.
                keyname_offset = BitConverter.ToUInt16(current_item, 0);
                data_type = current_item[3];
                value_size = BitConverter.ToUInt32(current_item, 4);
                value_size_padded = BitConverter.ToUInt32(current_item, 8);
                data_offset = BitConverter.ToUInt32(current_item, 12);

                // stuff for debug...
                //Console.WriteLine("keyname_offset:" + keyname_offset);
                //Console.WriteLine("data_type:" + data_type);
                //Console.WriteLine("value_size:" + value_size);
                //Console.WriteLine("value_size_Padded:" + value_size_padded);

                // lets get the key name.
                // not too hard, really. 

                string key_name = "";
                char cchar = '~'; // we need a char to screw with. Any char, but ~ is a nice one.
                uint kidx = key_offset + keyname_offset; // our index into the byte structure.
                // loop until we see a NULL, as thats our seperator char.
                while (cchar != (char)0x00)
                {
                    // get our current char.
                    cchar = (char)qq[kidx];
                    // Is it a control char?
                    if (char.IsControl(cchar))
                    {
                        // It is, go away.
                        break;
                    }
                    // add our current char.
                    key_name += cchar;
                    // and move to the next char.
                    kidx++;
                }
                //Console.WriteLine("Key:" +key_name);

                byte[] item_data = new byte[value_size_padded];
                item_data.Initialize();
                Array.Copy(qq, val_offset + data_offset, item_data, 0, value_size);

                Array.Resize(ref item_data, (int)value_size);

                SFOEntry ent = new SFOEntry();
                ent.type = (SFOEntryType)data_type;
                ent.data = item_data;

                Dict.Add(key_name, ent);

            }



        }

        private static UInt32 RoundUpToMult(UInt32 input, UInt32 num)
        {
            return input + (num - (input % num));
        }


        public byte[] FormatData()
        {


            // this is how many bytes into the key table that the KEY is in.
            UInt16 key_offset = 0;
            // this is how many bytes into the VALUE table that the VALUE is in.
            UInt32 value_offset = 0;

            System.IO.MemoryStream keystream = new System.IO.MemoryStream();
            System.IO.MemoryStream datastream = new System.IO.MemoryStream();
            System.IO.MemoryStream tablestream = new System.IO.MemoryStream();
            int idx_key = 0;
            int idx_val = 0;
            foreach (var item in Dict)
            {
                // things we need to know.
                int padded_size = (int)RoundUpToMult((UInt32)item.Value.data.Length, (uint)4);
                byte[] key = System.Text.Encoding.UTF8.GetBytes(item.Key);
                byte[] dwrite = item.Value.data;


                // write our entry into the table.
                tablestream.Write(BitConverter.GetBytes((UInt16)idx_key), 0, 2); // offset of key name into the key table.
                tablestream.WriteByte(4); //unknown '4' byte.
                tablestream.WriteByte((byte)item.Value.type);
                tablestream.Write(BitConverter.GetBytes((UInt32)item.Value.data.Length), 0, 4);
                tablestream.Write(BitConverter.GetBytes((UInt32)padded_size), 0, 4);
                tablestream.Write(BitConverter.GetBytes((UInt32)idx_val), 0, 4);


                keystream.Write(key, 0, key.Length);
                keystream.WriteByte(0x00); // write a null byte, to be safe.
                idx_key += key.Length + 1;


                Array.Resize(ref dwrite, padded_size);

                datastream.Write(dwrite, 0, padded_size);
                idx_val += padded_size;
            }

            //byte[] return_array = new byte[20+info_table.Length + keystream.Length + datastream.Length];

            byte[] HEADER = { 0x00, 0x50, 0x53, 0x46, 0x01, 0x01, 0x0, 0x00 };

            //Array.Copy(HEADER, return_array, 8);
            //Array.Copy(BitConverter.GetBytes((UInt32)(20 + tablestream.Length)), 0, return_array, 8, 4);
            //Array.Copy(BitConverter.GetBytes((UInt32)(20 + tablestream.Length + keystream.Length)), 0, return_array, 12, 4);
            //Array.Copy(BitConverter.GetBytes((UInt32)Dict.Count), 0, return_array, 16, 4);
            //Array.Copy(tablestream.ToArray(), 0, return_array, 20, tablestream.Length);
            //Array.Copy(keystream.ToArray(), 0, return_array, 20 + tablestream.Length, keystream.Length);
            //Array.Copy(datastream.ToArray(), 0, return_array, 20 + tablestream.Length + keystream.Length, datastream.Length);


            System.IO.MemoryStream ms = new System.IO.MemoryStream();
            ms.Write(HEADER, 0, 8);                 /* test to see if something is just broken... */
            ms.Write(BitConverter.GetBytes((UInt32)(/*20 + */ 20 + tablestream.Length)), 0, 4);
            ms.Write(BitConverter.GetBytes((UInt32)(/*20 + */ 20 + tablestream.Length + keystream.Length)), 0, 4);
            ms.Write(BitConverter.GetBytes((UInt32)Dict.Count), 0, 4);
            ms.Write(tablestream.ToArray(), 0, (int)tablestream.Length);
            ms.Write(keystream.ToArray(), 0, (int)keystream.Length);
            ms.Write(datastream.ToArray(), 0, (int)datastream.Length);

            return ms.ToArray();

        }

        // our internal sorted dictionary. 
        SortedDictionary<string, SFOEntry> Dict;

        #region ICollection<KeyValuePair<string,SFOEntry>> Members

        public void Add(KeyValuePair<string, SFOEntry> item)
        {
            Dict.Add(item.Key, item.Value);

        }

        public void Clear()
        {
            Dict.Clear();
        }

        public bool Contains(KeyValuePair<string, SFOEntry> item)
        {
            return Dict.Contains(item);
        }


        /// <summary>
        /// UNUSED!
        /// </summary>
        /// <param name="array"></param>
        /// <param name="arrayIndex"></param>
        public void CopyTo(KeyValuePair<string, SFOEntry>[] array, int arrayIndex)
        {
            return;
        }

        public int Count
        {
            get { return Dict.Count; }
        }

        public bool IsReadOnly
        {
            get { return false; }
        }

        public bool Remove(KeyValuePair<string, SFOEntry> item)
        {
            return Dict.Remove(item.Key);
        }

        #endregion

        #region IEnumerable<KeyValuePair<string,SFOEntry>> Members

        public IEnumerator<KeyValuePair<string, SFOEntry>> GetEnumerator()
        {
            return Dict.GetEnumerator();
        }

        #endregion

        #region IEnumerable Members

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            return Dict.GetEnumerator();
        }

        #endregion

        #region IDictionary<string,SFOEntry> Members

        public void Add(string key, SFOEntry value)
        {
            Dict.Add(key, value);
        }

        public bool ContainsKey(string key)
        {
            return Dict.ContainsKey(key);
        }

        public ICollection<string> Keys
        {
            get { return Dict.Keys; }
        }

        public bool Remove(string key)
        {
            return Dict.Remove(key);
        }

        public bool TryGetValue(string key, out SFOEntry value)
        {
            return Dict.TryGetValue(key, out value);
        }

        public ICollection<SFOEntry> Values
        {
            get { return Dict.Values; }
        }

        public SFOEntry this[string key]
        {
            get
            {
                return Dict[key];
            }
            set
            {
                Dict[key] = value;
            }
        }

        #endregion
    }
}

If this looks really horrible, its because the boards mess up the code.

Here (for you people who dont care about the above but want nice pretty-printed code) is a nice pretty-printed version: http://sonof.bandit.name/files/Ebootr2/sfofile.html

[slasher101]

// the filetype. NOTE!!! this isnt really useful. 
        public enum SFOFileType
        {
            SaveGame,
            MSGame,
            UMDGame

        }

LOL.

There's also no point doing:

public struct SFOEntry
        {
            public SFOEntryType type;
            public byte[] data;

            public SFOEntry(SFOEntryType t, byte[] d)
            {
                type = t;
                data = d;
            }

            public SFOEntry(string value)
            {
                type = SFOEntryType.txt;
                data = System.Text.Encoding.UTF8.GetBytes(value);
            }

            public SFOEntry(UInt32 value)
            {
                type = SFOEntryType.num;
                data = BitConverter.GetBytes((UInt32)value);
            }
            public SFOEntry(byte[] value)
            {
                type = SFOEntryType.bin;
                data = value;
            }

This, when the 2 variables you are writing to are public. You only need this if they are private members.

[indrora]

@slasher: There's a Really Good Reason(tm) for having overloaded constructors.

Because I have those three overloaded constructors, I can go like this:

mySfoFile.Add("TITLE", new SFOEntry("What the hell?"));

Its for pure flexibility. There's some 21 different ways to show a MessageBox in C# -> MessageBox.Show Method (System.Windows.Forms)

Stucts in .NET are also finnicky things. They're like classes but a lot lighter on the brain. They also always have a default constructor, and its a *recommended practice* that you add overloaded constructors for structs.

Hell, the basic layout for a system.drawing.point is

struct Point
{
  public int x,y;
  public Point(int x, y) { this.x = x; this.y = y; }
  public Point(int x) { this.x = x; this.y = x; }
  public Point(Size s) { this.x = s.Width; this.y = s.Height; }
  public static ==(point a,point b) { return (a.x == b.x) && (a.y == b.y); }
}

Given that its a Common Practice to do such things (even in Classes!) as these are what are known as Pseudo-Named Parameters (Instead of going "t=SFOEntryType.bin, d={1,2,3,4,5}" they are specified in an order) and Easy Shortcuts. Its also *faster* in IL.

Here's what it looks like how *most* people use structs: (and classes for that matter!)

let ld0, null     // make sure our object is null first.
call .SFOEntry // Call its constructor. Note that we arent Pushing it onto the stack because we have no extra variables.
push lr0, ld0        // make sure the constructor gets pushed.
mov SFOEntryType->string,lo0 // 
mov "MY Title",ldstr0
push lo0, ld0~>SFOEntry_t
mov ldstr0, ldarg0
pop ldarg0
call System.Text.Encoding.UTF8.GetBytes(ldstr)
push lr0
mov lr0, ldstr0
push ldstr0,ld0~>SFOEntry_d

and this is what it looks like using overloaded constructors.

push "My Title",ldarg0
call .SFOEntry(string)
mov lr0, ld0

See how much more succinct and fast that is? the IL runtime will grab that code and start caching how it runs, making it run *faster*

Plus, its smart enough to know that if I screw up and use (Uint32) instead of (String), It'll find the right constructor (and throw a big warning) otherwise it will tell me "Hey dipstick! you didn't give a valid constructor!"

Overloaded structs are beautiful things in C#. Now, I do completely ignore them some times in the code but I make wonderful use of them throughout my code.

Also, yeah, the SFOFileType enum can be thrown away, it was just that I was using that as a "I'm walking through my documentation and wanted everything." paranoia.