Data Storage

[mattheweston]

I am currently designing the data storage for my game and some of my data may be best stored in a linked list; however, I am also looking into storing other data in text files to keep my data files as small as possible. Ideally, I would use a database to store this data, but due to overhead and file size I need a better way of doing this. Does anyone know of a good way to store data?

For example, I have a "league" that will contain data for the teams in the league as well as statistics and a schedule and league settings(including structure of the league itself). I am trying to figure out the best way to store my data and still try to maintain a small file size.

[JamesParkes]

I would think your text file method would be fine. You would just need a method for how to parse through your data files when reading them then, as well as a method for how to write to the files.

[mattheweston]

Ok I guess a better way of rewording the "question" would have been what is the best way to store a linked list in a text file. =)

[szdarkhack]

Ok I guess a better way of rewording the "question" would have been what is the best way to store a linked list in a text file. =)

If all your list nodes have 1 parent and 1 child you can just put the data in order with some sort of separator between nodes and lists. Now, if your list is more like a graph, you can simply add identifiers to "point" to the parents and/or children of each node. It would be even better in a binary format, of course, but if you want the files to be readable, you can go with simple text.

Your parser would then read the nodes, create the appropriate structures in memory and then set the pointers by parsing the identifiers. Here's a simple example of such a format (simple text):

List1:<1>Item1Data[2,3]\<2>Item2Data[]\<3>Item3Data[4]\<4>Item4Data
List2:<1>Item1Data[2]\<2>Item2Data[]

Here, i use '\\' as a separator between nodes and '\n' between lists. <number> is the node ID and [number_list] is a coma separated list of children IDs. This is of course just an example but you can do something like this for your needs. The parser will then read List1, create the nodes in memory and then assign the pointers to create the list structure, then repeat for List2 etc until EOF.

I hope this gives you some ideas

[mattheweston]

I was actually probably going to do binary files, but it would be "text files" writen in a binary format if that makes sense.

[Ginto8]

When dealing with putting a linked-list into a file, you must answer the question: what does a linked list represent? Linked lists represent a sequential set of data, sometimes doubly linked, sometimes linked in a circular fashion, but all in all, they are sequences. With this in mind, why not store them as you would an array? Their are a few reasons to use a linked list over an array in memory (faster add/remove, no requirement of large amounts of contiguous memory), but when writing to a file, why not let the filesystem deal with all that for you? Say your linked list was a set of characters: 'A' -> 'C' -> 'b' -> '+'. In this case, it's much more efficient to store it as "ACb+" than to try to replicate the whole data structure in the file.

When putting most data structures into files, there's a tendency to "flatten" them. In short, the actual structure parts are mostly removed, with only the data they stored and whatever little bits of information are required to remake the structure remaining. For example, a map implemented as a red-black tree may be flattened as a sequence of key-value pairs, and reconstructed into the tree on loading the file. This usually turns out better in the long run than trying to actually put a data structure into a text file... unless you're trying to create a filesystem, in which case creating data structures within a file is actually important.