Object Serialization in C#

[tappatekie]

Hey guys, just thought you would be interesting in checking out and using my Serializer class written in C#, the source code for it is attached. However, it is still a work in progress but I will not be updating this thread with code updates so if you want any features, you would have to code it yourself but I will be more than happy to see any amendments or additional features you have put in or request.

It allows you to serialize objects in C# to a binary representation (byte[]) and convert it back again, it also comes with the option for deep and shallow cloning (which takes advantage of the Serialize and DeSerialize functions).

Here is the full list of features:
1) Support for sbyte, byte, short, ushort, int, uint, long, ulong, string, char, bool, double, float and decimal serialization (however precision numbers are a bit buggy)
2) Enum serialization
3) DateTime object
3) 1 dimensional array serialization (cannot be jagged)
4) Additional serialization systems can be easily added so this class acts as the center of the entire serialization process
5) CLR objects (via Reflection)
6) Option for deep or shallow serialization as well as Clone, DeepClone and ShallowClone methods

[dandymcgee]

Not to undermine your work, but are you aware of the XmlSerializer class?

If you weren't, it may be worth comparing your implementation to Microsoft's. Theirs has some known limitations that yours may overcome. A performance benchmark might also be interesting.

[tappatekie]

Not to undermine your work, but are you aware of the XmlSerializer class?

If you weren't, it may be worth comparing your implementation to Microsoft's. Theirs has some known limitations that yours may overcome. A performance benchmark might also be interesting.

I'm aware of XML serialization but this is generally for a byte stream that is as small as possible to send across networks or save to disk (this was my primary focus, not performance)

Here are the benchmark tests using BinaryFormatter, XMLSerializer and my own Serializer (each is an average latency of 10 calls)
Note, the framework I used is Microsoft .Net 2.0 (it may be even faster in later versions) and there is NO form of compression in use to cheat

Code: Select all

Serializing array with 10000 string entries
My serializer
 >> Stream size: 268898 bytes
 >> Latency: 17ms
Testing Microsoft XMLSerializer
 >> Stream size: 449045 bytes
 >> Latency: 15ms
Testing Microsoft BinaryFormatter
 >> Stream size: 298917 bytes
 >> Latency: 7ms
===================================================

Serializing array with 100000 string entries
My serializer
 >> Stream size: 2788899 bytes
 >> Latency: 165ms
Testing Microsoft XMLSerializer
 >> Stream size: 4589045 bytes
 >> Latency: 54ms
Testing Microsoft BinaryFormatter
 >> Stream size: 3088917 bytes
 >> Latency: 73ms
===================================================

Serializing array with 1000000 string entries
My serializer
 >> Stream size: 28888900 bytes
 >> Latency: 1522ms
Testing Microsoft XMLSerializer
 >> Stream size: 46889045 bytes
 >> Latency: 538ms
Testing Microsoft BinaryFormatter
 >> Stream size: 31888917 bytes
 >> Latency: 951ms
===================================================

As you can see, the speed of my system is slower than the other systems, BUT the output size is significantly lower, so your risking latency for bandwidth basically.
I will be working to make this more faster though as I'm working to optimize the code as much as possible.

Here is the code I written to perform the benchmark (benchmarking the code I attached on the first thread): http://pastebin.com/YzYsN6WX

HOWEVER, it's slower for arrays yes, but for strings it's faster than binary serialization! and it's the same case again with output size.

Code: Select all

Serializing string with 10000 characters
My serializer
 >> Stream size: 10004 bytes
 >> Latency: 0ms
Testing Microsoft XMLSerializer
 >> Stream size: 10040 bytes
 >> Latency: 0ms
Testing Microsoft BinaryFormatter
 >> Stream size: 10025 bytes
 >> Latency: 1ms
===================================================

Serializing string with 100000 characters
My serializer
 >> Stream size: 100005 bytes
 >> Latency: 0ms
Testing Microsoft XMLSerializer
 >> Stream size: 100040 bytes
 >> Latency: 0ms
Testing Microsoft BinaryFormatter
 >> Stream size: 100026 bytes
 >> Latency: 1ms
===================================================

Serializing string with 1000000 characters
My serializer
 >> Stream size: 1000005 bytes
 >> Latency: 4ms
Testing Microsoft XMLSerializer
 >> Stream size: 1000040 bytes
 >> Latency: 4ms
Testing Microsoft BinaryFormatter
 >> Stream size: 1000026 bytes
 >> Latency: 4ms
===================================================

If you guys want, I will benchmark serializing an object with 10,000, 100,000, and 1,000,000 properties and fields (with ints, strings and arrays combined)

[dandymcgee]

Nice results, thanks for that. I actually wasn't thinking all that much about the fact that XML is obviously going to be much larger than a byte stream as I was thinking more of save files. Byte stream is obviously the right choice for serializing objects to be sent over a network.

Any idea why BinaryFormatter produces slightly larger results than your serializer? What other data is it storing?

[tappatekie]

I looked at the BinaryFormatter output and I'm guessing it's encoding information and size/type information so that you deserialize into the correct type safely. If it's true, it's unwanted information since my serializer trusts you to know what you are deserializing.

It could also be Microsoft's approach to writing a string value since you first need to specify the strings length and their approach is probably the one that uses up a few bytes on the stream.

This is my formatting of serializing the length of the string:
[byte 0] length of the binary bytes that contain the length
[byte 1....] 8 bit chunks of the length
An example would be, 256 would be encoded as (2, 1, 255) and 32767 (16bit max) would be (2, 255, 255)

[CC Ricers]

If the object is to send data across networks or save it to disk, it would be a good idea to apply some sort of Runtime-Length Encoding to your serializer as an option. That way you can compress groups of similar contiguous bytes to a smaller space.

For example XXXXXYYYZZ can be converted to 5X3Y2Z in byte data. You just saved 4 bytes. Two bytes per data type may incur overhead if the data changes very frequently, but some compression algorithms would be good enough to know whether the extra byte is needed to encode it.

for example, a PCX format-like RLE can let you store any one of 192 ascii values and the number of times it repeats (up to 64 times), if it has to repeat at least 3 times.

The byte values determine what comes ahead. If the two most significant bits are 1 this byte stores a run of identical data.
0x00-0xBF: this is data encoded directly (a value from 0-191)
0xC0-0xFF: RLE run (1-64 bytes), followed by byte