What Have You Done : 6th June 2012/Forum (SoCoder)

Forum Home

Wednesday Worklogs

What Have You Done : 6th June 2012

User

Message

Posted : Wednesday, 06 June 2012, 13:55 | Permalink | Mark Here

JL235

WW Entries : 7

It's been a while since we last had one of these, sort I thought I'd resurrect it. What have you been up to over the last week? Or even longer? What projects are you working on, how are they coming along? What has gone well, what has gone badly, and what do you have planned for next week? ----- PlayMyCode.com - build and play in your browser, Blog, Twitter.

Posted : Wednesday, 06 June 2012, 14:53 | Permalink | Mark Here

Afr0

WW Entries : 3

Still working on Project Dollhouse.
Lately I've been working on support for *.iff, *.spf, *.flr and *.wll, which are variations on the same containerformat.
It is a slow process (really slow!), because the format (Interchangeable File Format) as implemented by Maxis is non-standard, self-contradictory and filled to the brim with gotchas.
Thanks to information graciously donated by original developer(s), we (me and Fatbag of NIOTSO) have enough information (hopefully) to piece together the VM running SimAntics code (IFF files contain BHAV chunks that each represent a function written in SimAntics).
However, rebuilding the Edith (EDiTh House) VM is a slow and tedious process of which we've barely scratched the surface.
So far I've implemented the sleep and idle commands.
Right now I'm working on replacing the ugliest parts of the IFF reading code with better code based on more up-to-date information. Once I'm done with that, the plan is to either finish 3D rendering once and for all, or figuring out how to decompress SPR# sprites (Maxis had (and still has, based on newer games like The Sims 2) a habit of implementing their own insane compression algorithms, breaking their own standards and doing stuff like using every way of representing a string known to mankind in their formats).

Some examples of what I'm talking about:

        /// <summary>
        /// Because they couldn't decide on a string length, some guy apparently thought
        /// it was OK to assume a string length of 64, and zero-terminate if the string
        /// happened to be shorter...
        /// </summary>
        /// <returns>The string read from a chunk-header.</returns>
        private string GetNameString()
        {
            char Chr;
            bool IsZeroTerminated = false;
            int i;

            for (i = 0; i < 63; i++)
            {
                Chr = (char)m_Reader.PeekChar();

                if (Chr == '\0')
                {
                    IsZeroTerminated = true;
                    break;
                }
            }

            if (IsZeroTerminated)
                return new string(m_Reader.ReadChars(i));
            else
                return Encoding.ASCII.GetString(m_Reader.ReadBytes(64));
        }

-->

        /// <summary>
        /// Reads the chunks of the IFF-archive by looking for the RSMP.
        /// </summary>
        private void ReadChunks()
        {
            string Identifier = new string(m_Reader.ReadChars(60)).Replace("\0", "");

            if (Identifier != "IFF FILE 2.5:TYPE FOLLOWED BY SIZE JAMIE DOORNBOS & MAXIS 1")
            {
                throw new Exception("Invalid iff file!");
            }

            /*byte[] b = m_Reader.ReadBytes(4);

            //Find the offset of the resourcemap (rsmp) chunk.
            int resMapOffset = b[0] | b[1] << 8 | b[2] << 16 | b[3] << 24;
            if (resMapOffset > m_Reader.BaseStream.Length || resMapOffset < 0)
                resMapOffset = b[3] | b[2] << 8 | b[1] << 16 | b[0] << 24;*/
            uint resMapOffset = Endian.SwapUInt32(m_Reader.ReadUInt32());

            Dictionary<string, List<uint>> files = new Dictionary<string, List<uint>>();

            if (resMapOffset != 0)
            {
                long pos = m_Reader.BaseStream.Position;
                m_Reader.BaseStream.Position = resMapOffset;

                m_Reader.BaseStream.Position += 76; //Skip the header.

                m_Reader.ReadInt32(); //Reserved
                uint version = m_Reader.ReadUInt32();
                m_Reader.ReadInt32(); //pmsr
                m_Reader.ReadInt32(); //Size
                uint typeCount = m_Reader.ReadUInt32(); //How many types are present in this *.iff...

                for (uint i = 0; i < typeCount; i++)
                {
                    //NOTE: For some types in some files this is empty...
                    string typeCode = new ASCIIEncoding().GetString(m_Reader.ReadBytes(4));

                    if (version == 0)
                    {
                        //Empty RSMP...
                        //numEntries + 1 entry without label = 13 bytes.
                        if ((m_Reader.BaseStream.Length - m_Reader.BaseStream.Position) < 13)
                        {
                            files.Clear();
                            FuckThisShit(ref files);
                            break;
                        }
                    }
                    else if (version == 1)
                    {
                        //Empty RSMP...
                        //numEntries + 1 entry without label = 16 bytes.
                        if ((m_Reader.BaseStream.Length - m_Reader.BaseStream.Position) < 16)
                        {
                            files.Clear();
                            FuckThisShit(ref files);
                            break;
                        }
                    }

                    //How many entries there are...
                    uint numEntries = m_Reader.ReadUInt32();

                    List<uint> offsets = new List<uint>();
                    for (uint j = 0; j < numEntries; j++)
                    {
                        if (version == 0)
                        {
                            //Empty RSMP...
                            //Minimum size for an entry without a label is 9 bytes.
                            if ((m_Reader.BaseStream.Length - m_Reader.BaseStream.Position) < ((numEntries - j) * 9))
                            {
                                files.Clear();
                                FuckThisShit(ref files);
                                break;
                            }
                        }
                        else if (version == 1)
                        {
                            //Empty RSMP...
                            //Minimum size for an entry without a label is 12 bytes.
                            if ((m_Reader.BaseStream.Length - m_Reader.BaseStream.Position) < ((numEntries - j) * 12))
                            {
                                files.Clear();
                                FuckThisShit(ref files);
                                break;
                            }
                        }

                        uint offset = m_Reader.ReadUInt32();
                        m_Reader.ReadInt16(); //ChunkID
                        if (version == 1) { m_Reader.ReadInt16(); } //ChunkID
                        m_Reader.ReadInt16(); //Flags
                        if (version == 1)
                        {
                            byte Length = m_Reader.ReadByte();
                            if (Length > 0)
                                m_Reader.ReadBytes(Length);
                        }
                        else
                        {
                            GetNameString();
                        }
                        offsets.Add(offset);
                    }

                    if (!files.ContainsKey(typeCode))
                        files.Add(typeCode, offsets);
                }

            }
            else
            {
                List<KeyValuePair<string, uint>> offsets = new List<KeyValuePair<string, uint>>();
                while (true)
                {
                    uint offset = (uint)m_Reader.BaseStream.Position;

                    string tag = new ASCIIEncoding().GetString(m_Reader.ReadBytes(4));

                    /*byte[] bytes = m_Reader.ReadBytes(4);

                    if (bytes.Length == 0)
                        break;*/

                    //int size = bytes[3] | bytes[2] << 8 | bytes[1] << 16 | bytes[0] << 24;
                    uint size = Endian.SwapUInt32(m_Reader.ReadUInt32());

                    m_Reader.BaseStream.Position += (size - 8);

                    if(!tag.Equals("XXXX"))
                        offsets.Add(new KeyValuePair<string, uint>(tag, offset));

                    //76 bytes is the size of a chunkheader, so don't bother reading the next one
                    //the stream has less than 76 bytes left.
                    if (m_Reader.BaseStream.Position == m_Reader.BaseStream.Length ||
                        (m_Reader.BaseStream.Length - m_Reader.BaseStream.Position) < 76)
                        break;
                }

                List<string> typesFound = new List<string>();

                foreach (KeyValuePair<string, uint> kvp in offsets)
                {
                    if (!typesFound.Exists(delegate(string s) { return s.CompareTo(kvp.Key) == 0; }))
                    {
                        List<KeyValuePair<string, uint>> theseChunks = offsets.FindAll(delegate(KeyValuePair<string, uint> pair) { return pair.Key.CompareTo(kvp.Key) == 0; });
                        List<uint> offsetValues = new List<uint>();
                        foreach (KeyValuePair<string, uint> kvp2 in theseChunks)
                        {
                            offsetValues.Add(kvp2.Value);
                        }

                        if (!files.ContainsKey(kvp.Key))
                            files.Add(kvp.Key, offsetValues);

                        typesFound.Add(kvp.Key);
                    }
                }
            }

            foreach (KeyValuePair<string, List<uint>> file in files)
            {
                foreach (int offset in file.Value)
                {
                    if (offset > 0)
                    {
                        m_Reader.BaseStream.Position = offset;

                        byte[] Buf = m_Reader.ReadBytes(4);
                        string StrResource = Encoding.ASCII.GetString(Buf);

                        if (StrResource == "SPR#" || StrResource == "SPR2" || StrResource == "rsmp" || StrResource == "PALT" ||
                            StrResource == "DGRP" || StrResource == "STR#" || StrResource == "BHAV" || StrResource == "FWAV" ||
                            StrResource == "CTSS" || StrResource == "TTAB" || StrResource == "TTA" || StrResource == "OBJf" ||
                            StrResource == "BCON" || StrResource == "TPRP" || StrResource == "TMPL" || StrResource == "TRCN" ||
                            StrResource == "Optn" || StrResource == "SLOT" || StrResource == "GLOB" || StrResource == "FBMP" ||
                            StrResource == "BMP_")
                        {
                            //MessageBox.Show(StrResource);
                            IffChunk Chunk = ToChunk(StrResource, offset);
                            //i += (int)Chunk.Length;

                            m_Chunks.Add(Chunk);
                        }
                    }
                }
            }
        }

        /// <summary>
        /// An archive had an empty rsmp, so fuck trying to read it
        /// and read all the chunkheaders instead.
        /// </summary>
        /// <param name="files">A list to fill with typetags (resourcename) and offsets.</param>
        private void FuckThisShit(ref Dictionary<string, List<uint>> files)
        {
            //IFF header is always 64 bytes - make absolutely sure we're at the right position in the file!
            m_Reader.BaseStream.Position = 64;

            List<KeyValuePair<string, uint>> offsets = new List<KeyValuePair<string, uint>>();
            while (true)
            {
                uint offset = (uint)m_Reader.BaseStream.Position;

                byte[] TagBytes = m_Reader.ReadBytes(4);
                Array.Reverse(TagBytes);
                string tag = new ASCIIEncoding().GetString(TagBytes);

                /*byte[] bytes = m_Reader.ReadBytes(4);

                if (bytes.Length == 0)
                    break;*/

                //int size = bytes[3] | bytes[2] << 8 | bytes[1] << 16 | bytes[0] << 24;
                uint size = Endian.SwapUInt32(m_Reader.ReadUInt32());

                m_Reader.BaseStream.Position += (size - 8);

                if (!tag.Equals("XXXX"))
                    offsets.Add(new KeyValuePair<string, uint>(tag, offset));

                //76 bytes is the size of a chunkheader, so don't bother reading the next one
                //the stream has less than 76 bytes left.
                if (m_Reader.BaseStream.Position == m_Reader.BaseStream.Length ||
                    (m_Reader.BaseStream.Length - m_Reader.BaseStream.Position) < 76)
                    break;
            }

            List<string> typesFound = new List<string>();

            foreach (KeyValuePair<string, uint> kvp in offsets)
            {
                if (!typesFound.Exists(delegate(string s) { return s.CompareTo(kvp.Key) == 0; }))
                {
                    List<KeyValuePair<string, uint>> theseChunks = offsets.FindAll(delegate(KeyValuePair<string, uint> pair) { return pair.Key.CompareTo(kvp.Key) == 0; });
                    List<uint> offsetValues = new List<uint>();

                    foreach (KeyValuePair<string, uint> kvp2 in theseChunks)
                    {
                        offsetValues.Add(kvp2.Value);
                    }

                    files.Add(kvp.Key, offsetValues);
                    typesFound.Add(kvp.Key);
                }
            }
        }

-->

-----
Afr0 Games

Project Dollhouse on Github - Please fork!

Posted : Wednesday, 06 June 2012, 15:05 | Permalink | Mark Here

shroom_monk

WW Entries : 8

Was gonna do a blog post about all my exploits of the last 6 months at some point, but I think I'll leave it til after exams. Currently working on exams, preparing for exams, or procrastinating from exams. At this very moment, procrastinating from exams. ----- A mushroom a day keeps the doctor away... Keep It Simple, Shroom!

Posted : Wednesday, 06 June 2012, 17:37 | Permalink | Mark Here

HoboBen

WW Entries : 9

I'm also in an endless purgatory of exams & revision. /firstworldproblems ----- github

Posted : Thursday, 07 June 2012, 03:04 | Permalink | Mark Here

spinal

Slowly chipping away at web dev. I've been told I must learn some colour theory as I haven't any idea which colours look good together. Also added a third control scheme to my wired ios game controller, which now includes iCade, Keyboard and Xperia Play emulation (Xperia play untested). So now it is (should be) able to work on iOS, Android, Windows, Mac, Linux ----- 3DS - 1762-2660-3218