Generating and using code during runtime

[Rebornxeno]

Hey all es folk and non-es folk. For this project I'm personally working on I need a way to compile c++ code given to my application, link with the symbols in the source-file of my (currently running) application, and run the compiled code at any arbitrary moment. Starting this I realize that I have no current knowledge of how a compiler deals with the symbols from the high-level language or how a compiler really works. So I plan on making my own compiler-program-thigamabob that will either: return straight assembly that I can use directly in the application by some form of self modification, or compile into a dll which I can dynamically link and use somehow.

The end goal is to produce an application from which I can develop c++ code and change the application to use that code, all without ever restarting the program. This program is really starting to sound just like a compiler...Hmm..

Any of you space cowboys care to lend advice from your experience with something like this?

[superLED]

I'm not sure if I got your point, but wouldn't Lua scripting do the exact same thing for you? Changing variables and such at run-time?

[THe Floating Brain]

I'm not sure if I got your point, but wouldn't Lua scripting do the exact same thing for you? Changing variables and such at run-time?

I agree with superLED, a scripting language like Lua (and non-script orientated languages that are interpreted do this too) can be "interpreted" at run-time.
Either by just sending straight text to the interpreter (which usually can be contained within some library or API you can use), or the languages custom form of Assembly (like Java).

However if you have a specific reason for using C++ IN THEORY (and by theory I mean highly unlikely case), I do not know if this would work with just using one .obj, or if it will work by reading from the .exe itself you could read from a generated file (.obj, .exe) made by a small light weight compiler like minGW http://www.mingw.org/ and read the file into a string. When this is done some compilers come with a asm( string here ); function or something like what VC++ 2010 has

Code: Select all

__asm {
}

were you can put Assembly and it will execute it, the kind of Assembly this can handle varies.

There are also C++ interpreters out there such as geordi http://weegen.home.xs4all.nl/eelis/geordi/ which I know this website uses: http://codepad.org/ and idk much about the C-Shell but that seems like an interpreter.

Anyway this C++ inception is kinda a radical idea, there are lots of great languages (sometimes refereed to as scripting languages) that you can easily embed into your program and use in the way you are describing, additionally they are usually not very hard to learn. One very popular one for game development is Lua http://www.lua.org/ and to use C++ OO stuff with it you can use Luabind http://blog.nuclex-games.com/tutorials/cxx/luabind/, toLua++ http://www.codenix.com/~tolua/ , or swig (for more than just lua I believe) http://www.swig.org/tutorial.html .

There are also a lot of other great easy to learn languages out there for this sort of thing if you do not like lua, like one of my personal favorites Python (3.2.2 that is for me at least):
Python 3.2.2 Doc: http://docs.python.org/py3k/
to send C++ classes to it you can use Boost.Python: http://www.boost.org/doc/libs/1_49_0/libs/python/doc/

Another one I would suggest but have not used too, too much my self is Game Monkey Script; it does not require additional library's to send classes to it (i believe) and it is similar to C++:

GMS: http://www.somedude.net/gamemonkey/
Tutorial Part 1: http://www.gamedev.net/page/resources/_ ... rt-1-r2282
Tutorial Part 2: http://www.gamedev.net/page/resources/_ ... rt-2-r2296

I hope I helped

[Rebornxeno]

I get really confused when you talk about scripting, mainly because I have never used one and don't understand why it's needed. If you need something done, why not do it in the native code you are writing in? Why switch to something working on-top to do it? And lastly, don't they have to run some function already defined? I have absolutely no idea what will need to be changed before runtime, it's anything decided after the program is already executing. I'll have to google what they do..

I'm going to do this in c++ or c because I want it to be portable, fast, and tiny in size. I know this sounds funny, but I think i'm trying to make an OS lol.

[ismetteren]

I get really confused when you talk about scripting, mainly because I have never used one and don't understand why it's needed. If you need something done, why not do it in the native code you are writing in? Why switch to something working on-top to do it?

Because you don't have to deal with compiling and linking when using an interperted language. It sound like what you want actually is code working on top of your application, therefore a language that works on top of it makes sense.

I'm going to do this in c++ or c because I want it to be portable, fast, and tiny in size. I know this sounds funny, but I think i'm trying to make an OS lol.

Without having much information about what you are trying to do, it sounds to me like you are trying to "solve the general problem": http://xkcd.com/974/ . It sound like your program is too general, so general that it is just another layer of complexity on top of the computer that dosen't really do anything. Offcourse, since you haven't really stated what you are trying to do i might be wrong

[THe Floating Brain]

I get really confused when you talk about scripting, mainly because I have never used one and don't understand why it's needed. If you need something done, why not do it in the native code you are writing in? Why switch to something working on-top to do it?

Because you don't have to deal with compiling and linking when using an interperted language. It sound like what you want actually is code working on top of your application, therefore a language that works on top of it makes sense.

Sorry I confused you, ismetteren makes a good point, were you do not have to worry about compiling/linking. In addition it makes your application more "extensible" for example:

in C++ I may have a class "Game Object".

Code: Select all

class GameObject
{
   protected:
      float x, y, z;
      bool initialized, update;
   public:
      //Imaginary encapsulation.//

      //Typically how I set up my "Game Objects".
      virtual void Initialize() {
      }
      virtual void Update() {
      }
      virtual void Destroy() {
      }
      void ChangeState()
      {
         //Stuff.//
      }
};

Well this is great; right? Lets add a singleton.

Code: Select all

//This has importance later.//
class Camera
{
   //Stuff.//

   void MoveTo( float x, float y )
   {
      //Stuff.//
   }
};

//Our singleton.//
struct Engine
{
   Camera camera;
   std::vector<GameObject*> objects;
   void MainLoop()
   {
      unsigned int size = objects.size();
      for( unsigned int i = 0; i < size; ++i )
      {
         objects[i]->ChangeState();
         if( objects[i]->GetInitialize() == true )
            objects[i]->Initialize();
         else if( objects[i]->GetUpdate() == true )
            objects[i]->Update();
         else
         {
            objects[i]->Destroy();
            GameObject *obj = objects[i];
            objects[i] = NULL;
            objects.erase( objects.begin() + i );
            delete obj;
        }
   }
   void Add( GameObject *obj ){
      objects.push_back( obj );
   }
};

Okay if you noticed our "Game Object" class
doesn't really do anything, well if we want to make it do something
we can just inherit it and use the vector of GameObject pointers in the
singleton to call our overwrites of Initialize(), Update() and Destroy() right?

Code: Select all

class MyNewObject: public GameObject
{
   //Overwrites.//

   //Im just ganna show Update() cuz im lazy :-P
   void Update()
   {
      camera.MoveTo( x, y );
   }
};

Wait dont we have to set reference to the engine to get the camera and yada yada yada
just to move the camera? Well that is kinda a pain in the butt.

Why not just put everything in one place?
Why not just write one class and not worry about OOP for
the rest of your app development?
Why not do live debugging?
:-D

---Enter interpreted languages---

Our GameObject class could have been simplified by doing something like the following.

Code: Select all

#include <My Fake Interpreted Language API>

class GameObject
{
   //Protected if you choose to expand upon this class
   //which is not entirely necessary.//
   protected:
   float x, y, z;
   std::string init, update__, destroy;
   bool initialized, update;
   public:
      void RunScript( std::string script ) {
         fakeLang.SendToScript( this );
         fakeLang.DoFile( script );
      }
      virtual void Initialize() {
         //Stuff.//
         RunScript( init );
      }
      virtual void Update() {
         //Stuff.//
         RunScript( update__ );
       }
       virtual void Destroy() {
          //Stuff.//
          RunScript( destroy );
       }
};

No changes to the engine code needed!!!
Just call this method (and add it to the singleton);

Code: Select all

void Send() {
   fakeLang.SendToScript( this );
   fakeLang.SendToScript( &camera );
}

You could implement the exact same thing you
were going to do in native code much easier.

Code: Select all

//---Update.fakeScript---//


//Your game object.
//object
//The camera.
//camera
//The singleton.
//engine

//Call a method :

camera:MoveTo( object:GetX(), object:GetY() )

You can make more objects do different things with the same class (you would have to tell each one what script to run though, I use a simple ID system).

Code: Select all

//---Update2.fakeScript---//


//Makes the object move.//

if( object:GetX() < 100 ):
   object:SetX( object:GetX() + 1 )
else:
   object:SetY( object:GetY() - 1 )

These script's would be called every frame AND (duuuunananana) live debugging!
You could even tell the program to re-write these.
Any changes made to them are applied next cycle after they have been overwritten/saved!

Hope this was not too confusing XD

[dandymcgee]

I'm going to do this in c++ or c because I want it to be portable, fast, and tiny in size. I know this sounds funny, but I think i'm trying to make an OS lol.

Without having much information about what you are trying to do, it sounds to me like you are trying to "solve the general problem": http://xkcd.com/974/ . It sound like your program is too general, so general that it is just another layer of complexity on top of the computer that dosen't really do anything. Offcourse, since you haven't really stated what you are trying to do i might be wrong

This. We can guess all day long, or you could try explaining what exactly you're attempting to do and we could provide more useful feedback than wall-of-text guesses like above.

[THe Floating Brain]

I'm going to do this in c++ or c because I want it to be portable, fast, and tiny in size. I know this sounds funny, but I think i'm trying to make an OS lol.

Without having much information about what you are trying to do, it sounds to me like you are trying to "solve the general problem": http://xkcd.com/974/ . It sound like your program is too general, so general that it is just another layer of complexity on top of the computer that dosen't really do anything. Offcourse, since you haven't really stated what you are trying to do i might be wrong

This. We can guess all day long, or you could try explaining what exactly you're attempting to do and we could provide more useful feedback than wall-of-text guesses like above.

<sarcasm> eeeeh... </sarcasm>

But ya +1

[Rebornxeno]

Well dandy, if you have a hard time wondering what I'm trying to do when I already wrote a sentence stating exactly that, I guess I can't help ya bud!

Thanks for the info guys, I see now that what I'm trying to do is build a sorta script system into the program. Though I'd really like to do it in c, even if I have to go through that compile/link crap.

[avansc]

sort of a solution

http://elysianshadows.com/phpBB3/viewto ... ing#p56308

altho, i would just recommend angelscript

[Falco Girgis]

So... Basically you want to embed a compiler into your application. This is almost exactly what JIT compilation is.

But I can guarantee that this is nothing like what you actually want. You want a scripting language...

[Rebornxeno]

I am just a kid who doesn't know what he wants. I can't believe I haven't heard of jit before.... Well this is pretty much what I was thinking of.

I guess I wasn't very specific before, I apologize. The program I'm designing is geared toward AI learning, integrating a mix of symbolic and connectionist neural networks. I want the application to have the ability to modify itself anyway it sees fit to solve a particular problem (nothing too general). The part I'm working on now is the jit part, something I have no prior experience doing.
Oh and another goal was to get the program to run pretty fast, so script is kinda out of the picture.

[dandymcgee]

Historically, Lisp was designed exactly to fit your needs. Though it is no longer the primary choice, I think the language is often underestimated. Perhaps it would benefit you to look into it, and languages supporting similar constructs (storing programs as data, and executing data as programs). Although, I'm not sure there is any language that does it as pure as Lisp.

[Rebornxeno]

I never realized how spoiled I am working with c++ for windows all the time, all I gotta do is some #include and bam all the windows functions available to me. With lisp I have to make all these foreign function definitions and stuff... Anyone reading this know of a way to go around an os? As in, making a window without having to use the windows api. I even use the windows api just for accessing the screen buffer so I can't even do that to just draw straight to the screen.

[lotios611]

Sounds like what you want is a genetic algorithm.