data storage
Moderator: Coders of Rage
-
- Chaos Rift Junior
- Posts: 200
- Joined: Mon Feb 22, 2010 12:32 am
- Current Project: Breakout clone, Unnamed 2D RPG
- Favorite Gaming Platforms: PC, XBOX360
- Programming Language of Choice: C#
- Location: San Antonio,Texas
- Contact:
data storage
I'm trying to figure out what to use for storing the data in my game. It's going to be a football game and I need to store all the stats as well as rosters and league information. I am currently debating between the following three options.
1. Roll my own
2. Access DB
3. SQLite
The game is going to have to create files that are able to be copied and ran on multiple computers like saving game data to a memory card and then using that on someone else's console.
Anyone ever tried/had to use a db for their game? Thoughts?
1. Roll my own
2. Access DB
3. SQLite
The game is going to have to create files that are able to be copied and ran on multiple computers like saving game data to a memory card and then using that on someone else's console.
Anyone ever tried/had to use a db for their game? Thoughts?
-
- Chaos Rift Junior
- Posts: 200
- Joined: Mon Feb 22, 2010 12:32 am
- Current Project: Breakout clone, Unnamed 2D RPG
- Favorite Gaming Platforms: PC, XBOX360
- Programming Language of Choice: C#
- Location: San Antonio,Texas
- Contact:
- Falco Girgis
- Elysian Shadows Team
- Posts: 10294
- Joined: Thu May 20, 2004 2:04 pm
- Current Project: Elysian Shadows
- Favorite Gaming Platforms: Dreamcast, SNES, NES
- Programming Language of Choice: C/++
- Location: Studio Vorbis, AL
- Contact:
Re: data storage
...Roll your own.
- LeonBlade
- Chaos Rift Demigod
- Posts: 1314
- Joined: Thu Jan 22, 2009 12:22 am
- Current Project: Trying to make my first engine in C++ using OGL
- Favorite Gaming Platforms: PS3
- Programming Language of Choice: C++
- Location: Blossvale, NY
Re: data storage
This. There's no need for a database, you're better off making something yourself which does exactly what you need without all the unneeded SQL stuff which is slow and useless for a game.GyroVorbis wrote:...Roll your own.
There's no place like ~/
- lotios611
- Chaos Rift Regular
- Posts: 160
- Joined: Sun Jun 14, 2009 12:05 pm
- Current Project: Game engine for the PC, PSP, and maybe more.
- Favorite Gaming Platforms: Gameboy Micro
- Programming Language of Choice: C++
Re: data storage
When should you use a database then?LeonBlade wrote:This. There's no need for a database, you're better off making something yourself which does exactly what you need without all the unneeded SQL stuff which is slow and useless for a game.GyroVorbis wrote:...Roll your own.
"Why geeks like computers: unzip, strip, touch, finger, grep, mount, fsck, more, yes, fsck, fsck, fsck, umount, sleep." - Unknown
-
- Chaos Rift Cool Newbie
- Posts: 78
- Joined: Mon Feb 21, 2011 2:55 am
- Current Project: Aleios Engine
- Favorite Gaming Platforms: PC, Dreamcast
- Programming Language of Choice: C++
- Location: Melbourne, Australia
Re: data storage
When in need of storing massive amounts of data. MMO's typically use databases, and all the crap for that is handled server side.lotios611 wrote:When should you use a database then?LeonBlade wrote:This. There's no need for a database, you're better off making something yourself which does exactly what you need without all the unneeded SQL stuff which is slow and useless for a game.GyroVorbis wrote:...Roll your own.
In this case he could simply make an XML file, or even just a text file. Chuck the data in some format in either a text file or XML file and it can be copied to many machines with ease.
- dandymcgee
- ES Beta Backer
- Posts: 4709
- Joined: Tue Apr 29, 2008 3:24 pm
- Current Project: https://github.com/dbechrd/RicoTech
- Favorite Gaming Platforms: NES, Sega Genesis, PS2, PC
- Programming Language of Choice: C
- Location: San Francisco
- Contact:
Re: data storage
That's just one of a million different places a database is used. Databases have a ton of advantages over a simple XML file. I would recommend using a database for any project require a large, indexed data store. If you need to search through thousands of records for a particular one, XML will perform like shit compared to SQLite. If you just need to store settings for one user, a text database (XML, BIN, TXT, whatever format it may be) will be more suited to your project.Aleios wrote:When in need of storing massive amounts of data. MMO's typically use databases, and all the crap for that is handled server side.lotios611 wrote:When should you use a database then?LeonBlade wrote:This. There's no need for a database, you're better off making something yourself which does exactly what you need without all the unneeded SQL stuff which is slow and useless for a game.GyroVorbis wrote:...Roll your own.
In this case he could simply make an XML file, or even just a text file. Chuck the data in some format in either a text file or XML file and it can be copied to many machines with ease.
@mathewweston In your case it depends one what you're doing with the league / roster information. I'd assume you're just loading it up for a particular "level" in the game then using it from there. In that case XML would work just fine.
Falco Girgis wrote:It is imperative that I can broadcast my narcissistic commit strings to the Twitter! Tweet Tweet, bitches!
- Falco Girgis
- Elysian Shadows Team
- Posts: 10294
- Joined: Thu May 20, 2004 2:04 pm
- Current Project: Elysian Shadows
- Favorite Gaming Platforms: Dreamcast, SNES, NES
- Programming Language of Choice: C/++
- Location: Studio Vorbis, AL
- Contact:
Re: data storage
Unless you are making an MMO or other sort of game where data is stored on a server, there is absolutely no need to use something as large and slow as a SQL database in an indie game.
And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.
And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.
- szdarkhack
- Chaos Rift Cool Newbie
- Posts: 61
- Joined: Fri May 08, 2009 2:31 am
Re: data storage
Because it compresses wellGyroVorbis wrote:And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.
Sorry, i just had to say it On a serious note though, it depends on the situation. If you need to store a lot of data and only read them just a few times, xml works fine (for example, a configuration file). But for stuff that you need to load quickly (like tilemaps) you're definitely better off with a binary file. It will save you a lot of loading time.
- LeonBlade
- Chaos Rift Demigod
- Posts: 1314
- Joined: Thu Jan 22, 2009 12:22 am
- Current Project: Trying to make my first engine in C++ using OGL
- Favorite Gaming Platforms: PS3
- Programming Language of Choice: C++
- Location: Blossvale, NY
Re: data storage
I agree with Falco, really the only time you should be using a database is if the information is going to be pulled across the Internet aka an MMO. In your case it makes the most sense to just pack together a binary of what you need and then parse it out. You save space, you save time by not using SQL and having to construct queries to pull out small bits of data, and it fits your needs and nothing more or less.GyroVorbis wrote:Unless you are making an MMO or other sort of game where data is stored on a server, there is absolutely no need to use something as large and slow as a SQL database in an indie game.
And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.
I also agree about XML being overused. XML is really only useful when it comes to things that you'll be changing manually. XML is a markup language, it's useful for changing manually because it's easy to read. In your case, the only thing that should be modifying your files is your application/game and nothing else really.
I hate seeing people talk about how they made an XML file for their level editor or for whatever they needed. And then they have to go and parse through the XML elements to grab information either manually with matching the XML string and not actually using proper XML parsing, or getting an XML parser that can spit out an array that you can pull data from. All of that is really slow and pointless, honestly a huge waste of time.
If you want to prototype something, or play around, at least make a comma delimited text file or something, not full blown XML. Then later once you realize what you need in your files, you can create a file that you can read in and you wont have to worry about being shit deep in XML for all your files and now it comes time to make everything into a binary.
There's no place like ~/
- Falco Girgis
- Elysian Shadows Team
- Posts: 10294
- Joined: Thu May 20, 2004 2:04 pm
- Current Project: Elysian Shadows
- Favorite Gaming Platforms: Dreamcast, SNES, NES
- Programming Language of Choice: C/++
- Location: Studio Vorbis, AL
- Contact:
Re: data storage
XML compresses better than a custom, binary-based file packed to 100% efficiency with not a single wasted character? And what about having to decompress this? And who says you can't compress your already compressed binary file?szdarkhack wrote:Because it compresses wellGyroVorbis wrote:And I also think XML is over-used. It's expensive and slow as hell to parse. It's overkill in almost every scenario that I've seen it used... If you are using a Level Editor or Toolkit, why in god's name would you go with ANYTHING other than a straight binary file? It's WAY smaller and WAY faster. Without debate, the optimal solution. The entire point of a Level Editor is to prevent you from having to create assets by hand... which is the only reason you would ever use something like XML.
Sorry, i just had to say it On a serious note though, it depends on the situation. If you need to store a lot of data and only read them just a few times, xml works fine (for example, a configuration file). But for stuff that you need to load quickly (like tilemaps) you're definitely better off with a binary file. It will save you a lot of loading time.
And sure, XML works "fine" when you're storing a lot of data and only reading them a few times like a config file... but goddamn, including an entire XML parsing library to load one file seems like overkill to me. Call me old-fashioned and low level, but that's definitely not something I will agree with. XML definitely has its place in software development... but unless you don't have a Level Editor, I don't see it in game development...
THANK YOU. I think this is the pathological worst-case. What is the point of a Level Editor? To automatically generate assets for your game/engine. Why the hell would you ever mess with these files manually? You are gaining absolutely nothing good from using XML in this scenario. It requires an additional runtime dependency in both your engine and editor, slows down your loadtimes, and bloats the hell out of your files. You've added NOTHING useful.LeonBlade wrote:I hate seeing people talk about how they made an XML file for their level editor or for whatever they needed. And then they have to go and parse through the XML elements to grab information either manually with matching the XML string and not actually using proper XML parsing, or getting an XML parser that can spit out an array that you can pull data from. All of that is really slow and pointless, honestly a huge waste of time.
With a Level Editor, you have a prime opportunity to compress the hell out of your files and make everything as efficient and tightly-packed as possible.
- szdarkhack
- Chaos Rift Cool Newbie
- Posts: 61
- Joined: Fri May 08, 2009 2:31 am
Re: data storage
I was kidding about the xml compressability... It's just what "fans" say all the time and i'm sick of hearing it. Using it for configuration purposes is in my opinion perfectly reasonable if you want them to be human-readable, given that the amount/structure of data justifies it. For example, if you can do it with a name-value pair file then do it, it'll be faster. There are cases, however, when you might need to have a more complex, hierarchical structure in your stored data, in which case XML is a pretty fair choice. In general though, the simpler you can afford to make things, the faster/smaller they will be.
Again, i'm in no way implying or suggesting that people should use xml for game data, you're clearly better off with binary files there. I thought i made that clear, but here it is once more, just to make sure.
Again, i'm in no way implying or suggesting that people should use xml for game data, you're clearly better off with binary files there. I thought i made that clear, but here it is once more, just to make sure.
- LeonBlade
- Chaos Rift Demigod
- Posts: 1314
- Joined: Thu Jan 22, 2009 12:22 am
- Current Project: Trying to make my first engine in C++ using OGL
- Favorite Gaming Platforms: PS3
- Programming Language of Choice: C++
- Location: Blossvale, NY
Re: data storage
Yeah I knew you were joking.szdarkhack wrote:I was kidding about the xml compressability... It's just what "fans" say all the time and i'm sick of hearing it. Using it for configuration purposes is in my opinion perfectly reasonable if you want them to be human-readable, given that the amount/structure of data justifies it. For example, if you can do it with a name-value pair file then do it, it'll be faster. There are cases, however, when you might need to have a more complex, hierarchical structure in your stored data, in which case XML is a pretty fair choice. In general though, the simpler you can afford to make things, the faster/smaller they will be.
Again, i'm in no way implying or suggesting that people should use xml for game data, you're clearly better off with binary files there. I thought i made that clear, but here it is once more, just to make sure.
There's no place like ~/
- Falco Girgis
- Elysian Shadows Team
- Posts: 10294
- Joined: Thu May 20, 2004 2:04 pm
- Current Project: Elysian Shadows
- Favorite Gaming Platforms: Dreamcast, SNES, NES
- Programming Language of Choice: C/++
- Location: Studio Vorbis, AL
- Contact:
Re: data storage
I didn't.LeonBlade wrote:Yeah I knew you were joking.szdarkhack wrote:I was kidding about the xml compressability... It's just what "fans" say all the time and i'm sick of hearing it. Using it for configuration purposes is in my opinion perfectly reasonable if you want them to be human-readable, given that the amount/structure of data justifies it. For example, if you can do it with a name-value pair file then do it, it'll be faster. There are cases, however, when you might need to have a more complex, hierarchical structure in your stored data, in which case XML is a pretty fair choice. In general though, the simpler you can afford to make things, the faster/smaller they will be.
Again, i'm in no way implying or suggesting that people should use xml for game data, you're clearly better off with binary files there. I thought i made that clear, but here it is once more, just to make sure.
Myyy bad.