Tiki RSS feed for weblogs http://old.falconpl.org/wiki/tiki-blogs_rss.php?ver=2 Thu, 17 May 2012 21:52:08 +0100 Tiki CMS/Groupware via FeedCreator 1.7.2 http://old.falconpl.org/wiki/img/tiki.jpg tikiwiki logo http://old.falconpl.org/wiki/tiki-index.php en-us Blog moved http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=2&postId=35 http://www.niccolai.cc.
]]> jonnymind Sat, 07 Jun 2008 21:29:03 +0100 Functional programming and Bitwise operators change on the run http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=2&postId=34
However, we miss a relevant feature: automatic binding of symbols in lists. We have substituted that with pass-by-reference semantics, as in the following example:
val = nil
eval( .[times $val .[.[printl $val]]] )

This works, but all the symbols needed in the list evaluation must be declared in advance, and they cannot be "closed" (as in functional closure). This limit the potential of our model.

I have found a way to provide arrays with symbol tables, as if they were functions or objects; also, it theoretically possible to access an array by object accessor like this:
sequence = .[printl ]
sequence. = "Hello world"
eval( sequence )

This would be quite a leap forward in our model, as we could externally bind anything with a variable in a functional construct, including other functional sequences. Also, this would allow inspecting functional sequences after their evaluation, exploiting at the fullest the "impure" functional model we picked.

The problem was... how to declare a symbol so that it stays local to the array sequence? (There are other smaller problems as well, as deciding the scoping of the declared variables, but one problem at a time).

The character "&" came immediately to my mind, but it was busy for the bitwise "and" operator. After a fast check, I realized that I have used up all the available one-character operator-like symbols commonly available on computer keyboards.

So, I realized that occupying three important "symbols" as "&", "cap" and "|" for operations that are rarely used was far from wise. We won't probably be able to stack this latest evaluable lists symbol table idea in this version we're releasing, but as I want the language and the engine to be somewhat stable for a long period before starting the work on 0.8.12 (and then 0.9), I decided to change the binary bitwise operators into "&&", "||" and "cap cap"; also the unary "not" bitwise operator is changed to the tilde symbol "~".

I know this may create a bit of confusion in C programmers coming to Falcon, but bitwise operators are a very rare feature in scripting languages anyhow, so rare that they may probably be declared through pseudofunctions, as bAnd( a, b), and few would complain (if someone at all). This frees up "&" for our needs, and also frees up "cap" and "|" which can be used for a whole lot of interesting features. Operator combinations as our dot-bracket ".[" are a good way to increase the number of available symbols, so "cap" may be used to modify the meaning of a following operator, making room for more things.

In this way, we'll be able to add the "&var" support as a drop-in somewhere after the release, without having to make incompatible the previous scripts.




]]> jonnymind Fri, 16 May 2008 08:46:25 +0100 Last thoughts on version 0.8.10 http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=2&postId=33
This is because after the release I want to keep the core system stable for a while, and concentrate on the module development and on the site management. We need libraries, we need a more falconized site and we need more people joining. So, the activity in the next months will be that of adding libraries and advertising Falcon around.

This Piuma version addresses a series of issues both in the language and in the engine, and adds some very important feature which complete the support for the programming paradigms we support. I am quite happy with it, and I think it can be a good base to develop modules. Just, before going on with the rest, I wanted to prepare the core and the VM systems for the tasks needed between release of 0.8.10 and the starting of 0.8.12.

Version 0.8.12 shall be the last of the 0.8 series, and will complete the engine, the internal API and the API of core and RTL. It's tentative name is now "Condor", as the intention is that to take a look from afar, fly high and placidly to view the ground from high above. A complete and final review of the code before starting series 0.9, that is final optimization steps before 1.0.

So, in the time between the Piuma release and the starting of works on the Condor engine, I wanted to have this three features ready:
- Multithreading pluggable support.
- Internationalization support with i"" strings
- Hooking for Virtual File System providers.

We have achieved the first two items; now, the engine has a slot for atomic reference counting and global locking (the latter currently unused) that can be hot-swapped by loaded-in modules. Multithreading module sets in its own "lock providers" so that atomic reference counting and proper locking are available when MT is used, and are linked with MT libraries only through the plug-in module. The way the mechanism is done is currently not very elegant, but it works (well) and it can stand till we review all the code on the Condor release.

About the last point, Falcon will have Virtual File System support plugged in the engine. In the beginning, I thought to provide a map of VSF providers, generating Streams based on the URI they received, directly on the VM, as the service system. But working at the MT pluggable support, I realized that the best way to do that is to review the whole pluggable system and give the ownership and control of all the pluggable support to the ENGINE (that is, the main Falcon DLL loaded by the embedding program). As such, the review of this and other aspects (as the Engine data, the Service system and the new MT support), is delayed as the very first activity of the Condor release.

So, the release is complete as it is now. The only "code" missing is a small installer utility that I wanted to be ready by the time of release, but being a separate project and being dedicated to modules, we can work at that right after the release.


]]> jonnymind Sun, 11 May 2008 10:21:24 +0100 Putting falcon on-line http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=2&postId=32
From the chatlog of this night...

--

Ok, I write here what I have done, because it's better if we're in 2 or 3 to know that.
I have that cute hosting.
On a USA virtual hosting server (inmotion
They are cute, but I have a fairly old fedora core 6 with support for virtual machines compiled.
So, upgrading is a delicate matter, and you got to live with the software you have on.
The point is that... we have apache 1.3 there.
And I have developed our extension for apache 2. Docs for apache 1.3 are sparse and outdated, and developing a new module from scratch for it is madness; also the module plant is TOTALLY different, so there is no re-usable code that may be used...
long story short, we're not doing things on apache 1.3
But I have that cute server...
So, I compiled from source apr-1.2, apr-utils and apache-2.2
Apr-1 lives well with apr-0.9; they have all separated.
But on a redhat based distro, where apache is called httpd and all its paths are mangled, it's a bit of hell.
So, I have installed apache2 under /usr/apache2.
Then, I have apache 2 running on localhost:8080, and our front end apache 1.3 proxies /newsite to 8080.
Problem was that the official apache server for fedora doesn't come with the proxy module.
So I had to download the sources of apache 1.3, and compiled the proxy module in shared mode.
I have created a startup script called http2 in /etc/init.d that ensures that our apache2 is loaded at startup.
this is actually an advantage. falcon 0.8.9 (the experimental piuma) is... experimental. In this way, if we crash or messup the back-end server, the front-end with the old site is still active;
when all will be ready, we'll simply switch the proxy target from /newsite to / and we'll have everything up and running.
site home is in /home/falcon10/newsite, owned by falcon10
Index document have been redefined to index.ftd
the second apache2 process is completely dedicated to serving that area. Also, since we are running on 2 physical CPUs, we won't incour in particular performance hit due to the proxying.
I would recommend a complete backup of /etc/httpd/conf and /usr/apache2 to preserve this setting.
(we have auto-backups included in the hosting price, but one can never know).
Fine; now it's time to make it work. tomorrow, I will start to work on the site. A bit of basic content and user management are a must.
The boring thing is that I have to compile everything there; i.e. I have to recompile dbi, hoping that devel packages on site are compatible with ours.
And now, I get a bit of rest.
Goodnight!
--

]]> jonnymind Wed, 30 Apr 2008 21:46:52 +0100 Empty loop tests http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=2&postId=31
I stacked up many things in the main VM loop, as i.e. preemption in functional context, end-frame callbacks, and even the generic vector as stackl; and it was long since I did an empty loop speed test.

I run the following (pseudo) code:
a = 0
for num in [0:100000000]
   a = a + 1
end


Here are the results, using "time", for several scripting engines:

C: 0m0.257s  (gcc -O0)
LUA: 0m8.021s
Falcon: 0m9.776s
PHP5: 0m14.923s
Python: 0m22.576s
Perl: 0m27.068s
Ruby: 1m25.947s


Now; an empty loop test doesn't mean anything in term of overall performance, but it's nice to see we have one of the fastest VM loop around... when it's still heavily under-optimized!

]]> jonnymind Fri, 25 Apr 2008 19:00:10 +0100 VM Stack http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=2&postId=30 ?, but this is suboptimal by far. First of all, representing the stack as a long linear vector poses relocation problems when the depth of the stack grows too much; secondly, the access and copy of single elements is dead slow with respect to the two mov eXX asms that an item copy requires.

Since there was also the problem that last call parameter count was limited to one byte in the stack frame structure, I took that occasion to do a bit of experimentations. An efficient call stack is a must for a functional language, so I took a bit of time to think how to make it better.

I tried a per-frame structure which holds only the local stack frame, and uses a list of frames, one per call. That seemed a good idea, but creating an instance (also, not a light one) for each call is definitely not a way to go.

Also, I took some time to do some experiment with the longtests:
Falcon unit test package.

Version 0.8.9 (piuma)
2g: success. ( 11.749 secs, 2553408.801 ops/sec)
2j: success. ( 15.906 secs, 1886080.724 ops/sec)
2h: success. ( 11.938 secs, 2512983.749 ops/sec)

2g, 2j and 2h tests calls functions respectively with depth of 100, 1000 and 10000 recursions; the total calls in each of them is the same, only the maximum depth of the calls varies. This experiment shows that there is a big performance hit when depth of the stack goes beyond a critical mass; then the application (i.e. faltest) is given some extra memory, and the it even recovers performance in the following steps.

So, while embedded scripts won't probably never break the 128 pre-allocated items in the linear stack, this performance (2.5M calls per seconds, plus some opcodes in each call), seems adequate even for functional based heavier applications in this 0.8 step.

So, experiments on the VM stack are closed till 0.9, where we'll perform massive optimizations. In that occasion, the stack will be turned into a specific wrapped Item*, and we'll experiment again for optimal growth strategies.

I suspect both the call sequence and the access to stack performance can be easily doubled (each item is pushed in the stack through a memcpy instead of a "=" operator that would be optimized in two movs), and reads require extra checks, that is branches, that an Item* won't need).

]]> jonnymind Thu, 27 Mar 2008 10:49:08 +0100 How errors can be ... fun http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=2&postId=29
The point is that, running a funny test of this feature on a test that already loaded and displayed an image (our project logo :-) I run sporadically on this error:
RangeError RT0047 at sdl_logo.__main__:37(PC:272): Access array out of bounds (STV)
  Traceback:
   sdl_logo.__main__:37(PC:272)

The code generating this was...
   pixels = screen.pixels

   for y = 0 to 479
      for x = 0 to 639
         n = x + y * 640
         pixels[ n ] = 0xFFFFFFFF
      end
   end

There was no memory corruption; just, at a spot the value of N went mad. Further investigation shown that inspecting its value cleared the error.
I started thinking there was a problem in nested fors; in fact, using a for-in on range cured the problem. I wanted to remove the for loops for a long time now, and since we have ranges that may store 3 values now, that is one thing I am loosely considering. However, before removing this statement, I wanted to be sure there wasn't another error hidden somewhere else.

I discovered that changing the patterns of garbage collection cleaned the error too. In the beginning, I thought the GC was ripping too much memory, or messing it up. But the thing seemed so clean. Then, I had a look at the VM code:
        PUSH    639
        PUSH    0
        FORI    _label_304, $*x, 0
.line 36
_label_240:
        MUL     $*y, 640
        ADD     $*x, A
        LD      $*n, A
.line 37
        STV     $*pixels, $*n, 268435455
        FORN    _label_240, $*x

The LD $*n, A instruction is the one that does the trick, moving the value of the pixel index in the variable that is then used to store the data. I started to think that the GC collection loop destroyed the A register and put something in that. A fast check proved I was right; there was an object instance in A by the time STV was called. Also, saving A before the GC call and restoring it right after did the trick.

But, the Garbage Collector is not supposed to change registers, and if it did it, then the whole thing would have blown out. I had a suspect, but to confirm it I added a data watchpoint to the A register right before entering the GC and... I got it.

GC scans object properties using CoreObject?.getProperty(). SDLScreen is a reflective class, and this means its UserData.getProperty() method gets called instead. Surface UserData creates a small object to store the rectangle in a property. Apart the fact that the object was re-created when not needed, the function doing this small step was...
CoreObject *MakeRectInst( VMachine *vm, const ::SDL_Rect &rect )
{
   Item *cls = vm->findWKI( "SDLRect" );
   fassert( cls != 0 );
   CoreObject *obj = cls->asClass()->createInstance();
   RectToObject( rect, obj );
   vm->retval( obj );
}

And vm->retval() changes the A register. I am so accustomed to do falcon extension functions (that are void and return data to VM through this mechanism) that I didn't realize I wrote that code instead of the correct return obj. Moreover, the compiler (g++4.x) didn't even warn me about this non-void function not returning. The thing worked normally as, after this function returns, normally, the value that it returns is really stored in A and then passed to the script. Just, when this happened in the GC loop this turned out in a disaster.

Someone may think this means that Falcon code is frail (well, no VM can be handed without care) but none of this would have happened if just the compiler told me I wasn't returning anything.



]]> jonnymind Fri, 21 Mar 2008 10:53:17 +0100 Packaging hell http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=2&postId=28
This brings up another interesting "hellish documentation" topic. You remember I said the worst doc around was Apache? — well, that's still true, but debian packaging nears it.

The problem with Debian docs on packaging (and with other docs you may find around, as i.e. Ubuntu guides), is that they SEEM good. They are so verbose, pointy, rich, that you are leaded to think that they are good docs. But when you get to do things you learn soon that the glue is missing. The basic overall informations on how to do things and why things are done in such ways is nowhere to be found.

Possibly, they seemed obvious to the first ones that invented the system.

I think the very basic point is the "policy" document, which is a very pointy and pitchy document stating what a Debian package should be and what it must be avoided. However, those rules descend from an overall mystic knowledge that remains precluded to the most, until you learn it by osmosis living in the environment formed by other packagers.

From the "policy", a set of learn-by-doing examples are extrapolated by various sources, as I.e. the Ubuntu packaging wiki. They are good, but usually they are partial and fail in explaining everything but the trivial cases. Worst of all, they too fail to explain the reasons behind things, and why tools works the way they do, so it is rather impossible to extend the usage of the tools beyond the mere samples that are shown.

I agree that doing good packaging (and so, good distros) is an arcane art, but even arcane arts have their grimoires. Even an absurd mass of laws as the Italian Civil Code is based on principles that can be read in books and learned before hitting against the wall of the minute details.

Another lesson learned about how *not* to write docs.
]]> jonnymind Thu, 17 Jan 2008 13:25:44 +0100 PgSQL, MySQL and SQLite3 http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=4&postId=27 
class Person(nName, nDob) from DBIRecord
  _tableName = "people"
  _primaryKey = "id"

  id   = nil
  name = nName
  dob  = nDob

  function age()
    // return computed age from dob
  end
end

p = Person( "John", lastYear )
p.insert()
p.name = "Jim"
p.update()
p.delete()


In the last post, you also saw that you can query objects:

rs = db.query( "SELECT * FROM people" )
while ( let p = rs.fetchObject( Person() ) )
  > p.name, " is ", p.age(), " years old"
end
rs.close


So, check out the SVN copy of the DBI module and let us know what you think!
]]> jeremy_c Thu, 03 Jan 2008 05:12:42 +0100 DBI is becoming usable! http://old.falconpl.org/wiki/tiki-view_blog_post.php?blogId=4&postId=26 
class Person
  ... attrs ...
end

rs = dbh.query( "SELECT * FROM people WHERE name LIKE $1", "%Smith" )
while ( let p = rs.fetchObject( Person() ) )
  > p.name, " was born on ", p.dob
end
rs.close()


Or, how about this one, query expansion by name from an object?

p = Person( 12, "John", "2005-10-20" )
dbh.execute( "UPDATE people SET name=$name, dob=$dob WHERE id=$id", p )


Or, one more. Transactions...

tr = dbh.startTransaction()
try
   tr.execute( "SQL CODE" )
   tr.execute( "MORE" )
   tr.commit()
catch DBIError in err
   tr.rollback()
end
tr.close()


So, that's just a quick run down. A few more things have to take place before I start working on other DBI drivers:

  • A bit more error checking
  • dbh/tr insert/update methods (dbh.insert( tableName, dict ))
  • dbh/tr queryOne (dbh.queryOne( "SELECT max(id) FROM people" ))
  • unit tests

However... please use svn, do a check out and report what you find! Currently, the only available DBI driver for the DBI module is PostgreSQL, but soon you will see others crop up such as MySQL, Sqlite3, Firebird/Interbase, ODBC, MS-SQL and hopefully more!
]]> jeremy_c Sun, 30 Dec 2007 07:00:03 +0100