GirToBac

[TJF]

GirToBac is a tool to generate FreeBasic (FB) header files for libraries based on GObject. For these libraries a GOject-Introspection file (suffix *.gir) can get generated containing all information to create language bindings for high-level programming languages like JavaScript, Vala and others. The aim is to support polyglot application development.

GirToBac is the first approach to connect the FreeBasic programming language to this tool-chain for easy creating and up-dating FreeBasic header files of GObject based libraries (ie like GTK, GDA, GooCanvas, ...). A set of example headers is available here.

Find more information in the online-documentation (or download here for offline-reading).

The source code and some example files are available at

• http://www.freebasic-portal.de/download ... c-296.html

[AGS]

I read about gir some time ago and did not see it's potential. On the GNOME
site (and elsewhere) I got the impression gir was useful for creating bindings
for languages like Python, Perl etc....

To use object introspection for generating freebasic bindings is very interesting.
Gnome is so big that a tool like girtobac could give fb users access to a huge
amount of very usable libraries.

[TJF]

Glad to find a visionary!

Your impression was right. GNOME libraries are interesting. The GObject Introspection (GI) technique opens the door for a wide usage. And now, the GirToBac project opens the door for FB users as well. I'm sure, polyglot applications do have a big potential.

GI bindings are easy to use. Each binary (*.DLL or *.so) has exactly one header file. Dependencies get resolved in the headers. (On windows the user has to care about the binary installation.)

GI is used for compiler languages like Vala to generate bindings by using the *.gir files. And, as you mentioned, it's also used for interpreter languages like Python, Perl, Tcl or JavaScript to bind libraries at run-time by using *.typelib files, which is -- more or less -- a binary form of the *.gir file.

A further interesting point is that bindings either can get generated in a C-like style, as GirToBac does in the current version (see examples). For these headers the C-API documentation can be used directly. But the FB code is also C-stylish and the user has to care about the memory management.

On the other hand GI files can get translated in native objects. That's done for all of the above mentioned programming languages. In that case the user cannot use the C-API documentation directly (small adaptions have to be done). But the memory management gets done by the OOP features of the language. The code gets shorter, gets more readable and the risc of memory leaks get reduced a lot.

Example (switch cursor style while executing any operation in GTK)

C-style (currently used)

Code: Select all

SUB do_anything_time_consuming(BYVAL Parent AS GdkWindow PTR)
  VAR old = gdk_window_get_cursor(Parent)
  VAR cursor = gdk_cursor_new(GDK_WATCH)
  gdk_window_set_cursor(Parent, cursor)

  ' do anything ...

  gdk_window_set_cursor(Parent, old)
  gdk_cursor_unref(cursor)
END SUB

OOP-style (may be possible in future)

Code: Select all

SUB do_anything_time_consuming(BYREF Parent AS Gdk.Window)
  VAR old = Parent.cursor ' or: = Parent.get_cursor()
  Parent.cursor = Gdk.cursor(Gdk.CursorType.watch) 'or: Parent.set_cursor(Gdk.cursor(Gdk.CursorType.watch))

  ' do anything ...

  Parent.cursor = old ' or: Parent.set_cursor(old)
END SUB

Although GirToBac is prepared for the OOP style of GI bindings, unfortunatelly the OOP features in FB (version 0.90.1) aren't good enought. The most important misses are

• Inheritance allows just one parent. We need to additionally handle at least several interfaces in an object.
• We need named CONSTRUCTORs (like a member function, but also constructing a new object)
• We need EXPLICIT working for ENUM blocks (fbc errors when keywords are used as names, although they're in a separate namespace).

Other optimisation would be helpful (ie regarding NAMESPACE).

We also may discuss about integrating GI features in the fbc source code (or in an external extension modul), so we don't need external headers for GI libraries in future. Instead fbc can generate just the neccessary part of the binding at compile-time (from a binary *.typelib file -> ie preprocessor command #TYPELIB "Xyz-1.0"). This'd be faster, since fbc need not scan the complete headers. The binding may be optional in OOP style (#TYPELIB_OOP "Xyz-1.0"). In this case the OOP integration would be better than the wrapper version in a header file (ie. direct error handling, or calling a member function directly instead of calling the FB objects member function that finally calls the library function). Sooner or later we may replace all headers by *.typelib files ...

Anyway, currently we can just generate and use the C-styled headers. And we should do so and report problems, to optimize FBs GI tool-chain. Ie. meanwhile I succesfully used GI-headers in most of my projects. But I'm not happy with the cairo-1.0.bi header. It's just a dummy, because cairo isn't based on GObject and GI cannot be used to generate a complete header. I fix it by calling the original cairo.bi header from the GI version. Other users may find further downsides or real problems ...

[AGS]

I was wondering why girtobac uses a callback based XML parser (as opposed
to a parser that builds a DOM).

And I don't know exactly what kind of tags a .gir file can contain but I'd like
to know.

I found a couple of programs that contain info on the .gir format. There is
https://git.gnome.org/browse/gobject-in ... rwriter.py
and at that same address (/tree/giscanner) there are some more .py files
that contain some info on possible tags/attributes.

There is of course also info on the .gir format in girtobac. But what I am missing is
an overview of the file format. Something in a nice notation like relax NG or relax NG
compact notation http://en.wikipedia.org/wiki/RELAX_NG

Knowing what the format of a .gir file is would help with creation of
test - cases. If the format of the .gir file would be known it would
not be necessary to use a tool to generate test cases (creating an
XML file is not that hard).

[TJF]

My test cases were header files, which I tranlated and successfully used in my projects (trial and error).

But you're right, it would be good to know the format details. There's some uncomplete information at

• GObject Introspection Reference Manual

Keep in mind, GI is work in progress and the documentation isn't finished jet.

To develop GirToBac I had to study the source code you mentioned. Most information came from

• gobject-introspection-1.33.10/girepository/girparser.c

and its modules.

[AGS]

My test cases were header files, which I tranlated and successfully used in my projects (trial and error).

But you're right, it would be good to know the format details. There's some uncomplete information at

• GObject Introspection Reference Manual

Keep in mind, GI is work in progress and the documentation isn't finished jet.

To develop GirToBac I had to study the source code you mentioned. Most information came from

• gobject-introspection-1.33.10/girepository/girparser.c

and its modules.

I felt like I just could not stand by and let you write down the exact format of .gir files.
So I started creating the schema for the .gir format using info from actual
.gir files (and .py files and your program and etc....).

I am using relax ng (non compact format). An example
of what I got so far (I've got a lot more already):

Code: Select all

<element name="include">
    <attribute name="name">
      <text/>
    </attribute>
    <attribute name="version">
      <text/>
    </attribute>        
</element>
<element name="c:include">
    <attribute name="name">
      <text/>
    </attribute>
</element>

A bit of <!--comment --> will be added to the relax ng schema to
add some meaning to entries (some are self - explanatory but
others could do with a little explanation of what can be within the <text/>
tag).

[TJF]

Looks great! Good luck to get the project finished.

I'm sure, the GNOME project would be thankful if you could structure this information for their GObject Introspection Reference Manual. (BTW: the documentation of the *.typelib format isn't finished either.)

[AGS]

I'm almost finished writing down the gir - format.
Turns out RELAX NG allows for definition of grammars by means of
XML. The schema for GNOME Builder has been written using
a RELAX NG format grammar as well.

I found another project trying to use gobject introspection
to generate bindings for a well known language:
http://wiki.freepascal.org/gir2pascal
http://sourceforge.net/p/lazarus-ccr/sv ... ospection/

gir2pascal pretty much sums it up I think. Someone has tried (or is trying?) to do for
FreePASCAL what you are trying to do for FreeBASIC.

I'm sure, the GNOME project would be thankful if you could structure this information for their GObject Introspection Reference Manual. (BTW: the documentation of the *.typelib format isn't finished either.)

It's been a bit of a hit-and-miss kind of thing for me when it comes to contributing something to an open source
project. Sometimes I get no response when trying to contribute (I usually contribute by posting a bugreport or
similar) and sometimes the bug I found gets fixed right away.

For the foreseeable future I don't think I will be contributing anything to any open source project
other than projects that are directly related to the FreeBASIC project.

About the .gir file format. it contains some strange parts.

There is a <bitfield> tag but also an <enumeration> tag. Both tags
are processed in exactly the same way.

In the source code of the gobject introspection project I can see this duplication.

Same thing for tag <glib:boxed> and <record>. Nowhere in girwriter or any other
tool related to the gobject introspection project does the difference in tags between
glib:boxed and record lead to any different processing.

From your code (girtobac)

Code: Select all

CASE "enumeration", "bitfield"
  PRINT #.FnrBi, NL "END ENUM";
  g_markup_parse_context_pop(ctx)

"record" I could find (in girtobac) but not "glib:boxed" (omitted on purpose I assume?).

Perhaps the different tags for bitfield, enumeration etc... are there for
'future purposes'?.

Here is my grammar of .gir files thus far. I am only posting the toplevel grammar
rules (and a bit more) as the grammar itself is already over 177 lines in size
and I am not done yet (the result could be anything in size between 300 and 1000 lines).

Code: Select all

<grammar>
  <start>
    <ref name="gir-document"/>
  </start>
</grammar>

<define name="gir-document"/>
  <ref name="repository">
</define>

<define name="repository">
  <element name="repository">    
    <attribute name="xmlns"/>
    <attribute name="xmlns:c"/>
    <attribute name="xmlns:glib"/>
    <zeroOrMore>  
      <ref name="include"/>
    </zeroOrMore><!--include-->
    <element name="package">
      <attribute name="name"/>
    </element>
    <zeroOrMore>
      <element name="c:include">
        <!--the name of the include file (path included)-->
        <attribute name="name"/>
      </element>            
    </zeroOrMore><!--c:include-->
    <ref name="namespace"/>
</define>

<define name="namespace"/>
  <element name="namespace">
    <attribute name="name"/>
    <attribute name="version"/>
    <optional>
      <attribute name="c:identifier-prefixes"/>
    </optional>
    <optional>
      <attribute name="c:symbol-prefixes"/>
    </optional>
    <optional>
      <!-- comma separated list of shared libraries
      no path needed-->
      <attribute name="shared-library"/>
    </optional>
    <zeroOrMore>
      <ref name="alias"/>
    </zeroOrMore>
    <zeroOrMore>
      <!-- tags have no particular order.
      Member can be both a member of a bitfield and a member of a record.
      The callback tag can also occur inside a field tag
       -->          
      <interleave>        
        <choice>
          <ref name="bitfield"/>
          <ref name="boxed"/>
          <ref name="callback"/>
          <ref name="class"/>
          <ref name="constant"/>
          <ref name="enumeration"/>
          <ref name="function"/>
          <ref name="interface"/>
          <ref name="record"/>
          <ref name="union"/>
        </choice>
      </interleave>
    </zeroOrMore>
</define><!--namespace-->

One thing that is missing is the opening tag of a
.gir file

Code: Select all

<?xml version="1.0"?>

<?xml version="1.0"> could be

Code: Select all

  <element name="?xml">
  <attribute name="version"/>

but that would leave an unaccounted ? after "1.0".
A tag name cannot start with ? in XML.

A final question: the format of a .gir file is 'known'. It would be easy
to create a parser for a .gir file without relying on an external tool
(a .gir file does not contain expressions or other parts that can
be challenging to parse).

Why not create your own .gir parser? h2bi was created from scratch
and it performed very well. Why not do the same for girtobac?

[TJF]

Long post, need some time to study it. Thanks for the inspiration!

Some promptly answers:

Why not create your own .gir parser? h2bi was created from scratch
and it performed very well. Why not do the same for girtobac?

A user of GirToBac headers must have GLib installed on his box since this library is the base of all the headers. So having GLib as a dependency for GirToBac is no big deal, from my point of view.

One thing that is missing is the opening tag of a
.gir file

Code: Select all

<?xml version="1.0"?>

That's handled by the GLib parser.

There is a <bitfield> tag but also an <enumeration> tag. Both tags
are processed in exactly the same way.

Yes, exactly the same for FreeBasic.

An enumeration member may have any numerical value (0, 1, 2, 3, ...) while a bitfield member usually has just one bit set (1, 2, 4, 8, 16, ... -- but there are also combination members for convenience).

It may be handled different in programming languages that provide a one-bit datatype.

Same thing for tag <glib:boxed> and <record>. Nowhere in girwriter or any other
tool related to the gobject introspection project does the difference in tags between
glib:boxed and record lead to any different processing.

I guess the <glib:boxed> tag can be used for libraries not writen in C ?!?

"record" I could find (in girtobac) but not "glib:boxed" (omitted on purpose I assume?).

I didn't find such a tag in any header.

<!-- tags have no particular order.
Member can be both a member of a bitfield and a member of a record.
The callback tag can also occur inside a field tag
-->

Tags are ordered alphabetically (AFAIR). Arrays can also occur in field or type tags. Some more hints are missing here ...

It's been a bit of a hit-and-miss kind of thing for me when it comes to contributing something to an open source
project. Sometimes I get no response when trying to contribute (I usually contribute by posting a bugreport or
similar) and sometimes the bug I found gets fixed right away.

Yes, I gathered similar experiences. When your commit is at the right place at the right time, it gets handled immediately.

[AGS]

"record" I could find (in girtobac) but not "glib:boxed" (omitted on purpose I assume?).

I didn't find such a tag in any header.

In an earlier post I mentioned the difference between girwriter.c and girwriter.py.
girwriter.c has not been updated for over a year while girwriter.py has been
updated recently. Looking at the content of the .gir files I have (gtk3, gdk3)
I am pretty sure these files were generated by girwriter.py, not girwriter.c.

The Python code tag that tells me girwriter.py is the tool used
to generate .gir files looks like this

Code: Select all

if hasattr(node, 'doc') and node.doc:
  self.write_tag('doc', [('xml:space', 'preserve')],node.doc)

There is no mentioning of a doc node in girwriter.c Meaning girwriter.c
cannot write a doc tag and cannot be the producer of the .gir files
I am using (as those files contain doc tags).

In your message you refer to the tags being sorted. girwriter.c does no
sorting so xml produced by girwriter.c has no sorted tags.

But as pywriter.py is used to generate .gir files tags are sorted
as pywriter.py does sort tags.

Regardless of pywriter.py sorting tags I still find tags are not (always)
sorted in the .gir files I am using.

Example.

File starts with (only relevant attribute shown)

Code: Select all

<repository version="1.2">

Later on in the file I find a record tag followed by a bitfield tag
(at the same depth). record < bitfield making me think tags are not
(always) sorted.

I am going to stick with giwriter.c as it is very doable
to find out the .gir format by looking at the C code.
And because girwriter.c deals with attribute values like "GLib.List",
"GLib.SList" and "GLib.Array" in a special way.
girwriter.py just writes those attribute values without a change
(which explains how a .gir file can have both doc elements
and GList.Slist etc... attribute values).

Last but not least: the .gir format allows to add an arbitrary number
of attribute tags to an existing tag in a .gir file. The attribute tag that
can be added looks like

Code: Select all

<element name="attribute">
  <attribute value="name"/>
  <attribute value="value"/>
</element>

Using the attribute tag it would be possible to extend the .gir format.
An example usage could be a tag to denote some platform specific value

Code: Select all

<attribute name="platform" value="linux"/>
<attribute name="include_path" value="some_path"/>
<attribute name="platform" value="windows"/>
<attribute name="include_path" value="some_other_path"/>

Tags have a set sequencing so if there is an attribute tag then a check
for attribute value platform would be enough to continue parsing
either linux - related info or windows related info. Whether this kind
of use of the attribute tag is really usable I don't know. But at
least it will be possible to extend girtobac without breaking the
parsing of a .gir file that does not contain attribute tags
(at the moment girtobac does not recognize attribute tags).

Is there a reason for girtobac not handling some array related
attributes/attribute values in a specific manner?
(attribute "zero-terminated", attribute values "Glib.Array",
"Glib.PtrArray", "GLib.ByteArray").

[TJF]

Thanks AGS, for analysing GirToBac and its environment.

Is there a reason for girtobac not handling some array related
attributes/attribute values in a specific manner?
(attribute "zero-terminated", attribute values "Glib.Array",
"Glib.PtrArray", "GLib.ByteArray").

A zero-terminated array need no specific handling in the header (since it's just a pointer to the start of the array). The user has to handle it in a specific manner in the source code.

Speaking about c-styled header generation GLib.Array, GLib.List and GLib.SList need an additional PTR in the declaration, unlike Glib.PtrArray. I'll check Glib.ByteArray, I'm unsure ATM.

(at the moment girtobac does not recognize attribute tags).

ATM there is no attribute tag in any *.gir file I translated.

It's good to know about this tag. But it's not enough to recognize it, it needs the right logic to handle it as well. And the logic is variable from tag to tag.

I think I'll make GirToBac emitting a message when such a tag gets recognized (I guess it only occurs at top level ?!?).

Later on in the file I find a record tag followed by a bitfield tag
(at the same depth). record < bitfield making me think tags are not
(always) sorted.

I found the tags sorted by element name (not by tag type). But I may be wrong and mix it with the *.typelib format. There the elements are sorted by name, full sure.

There is no mentioning of a doc node in girwriter.c Meaning girwriter.c
cannot write a doc tag and cannot be the producer of the .gir files
I am using (as those files contain doc tags).

Yes, that's my impression too. girwriter.py is part of girscanner (which is used to generate the gir files) and girwriter.c is part of the libgirepository (which can be used to parse or generate any GI files in a binary mode -- without documentation).


My general impression is that you try to make a perfect GirToBac tool. Sorry, rather than making a perfect tool, my aim is to get perfect headers in an easy way. New headers as well as up-dates.

While I developed GirToBac the *.gir-format changed two times. It looks like a moving target.

And how often do we use GirToBac? Two or three times a year. I thought about auto-sorting the elements (and removing the <first/> tag in the control file). But it's not worth the effort since sorting is a bit complex. And the manual sorting can be reused for up-dates, so it needs to be done just once.

I look forward to get some feedback on the usability of the headers (without macros). And the next main target is to add the OOP styled translation as soon as possible, but this needs some improvements in the fbc (working around the missing features will increase the compilation time for the headers and needs a lot of name mangling for ENUM members, which needs a lot of additional documentation).

[TJF]

Update available:

• New: GirToBac source 32/64 bit ready.
• New: Translated header code 32/64 bit ready.
• New: Self translated GLib header in use, see [url]ttp://downloads/bibliotheken/gtk-3-header-dateien-fuer-freebasic-191.html]GnomeHeader Set[/url]).
• New: Configuraton file attributes binary, check and pack.
• New: Caller / callee graphs in the documentation.
• Bugfix: Gir attribute "throws" now recognized in all functions.
• Bugfix: String literals (instead of !@"..." now correct @!"..." is used).
• Bugfix: Minor adaptions in the documentation.

Download:

• GirToBac-0.2