Struggles building Langkit with Alire

[cbourgeois]

Bonsoir, hope that you're all doing well.

I am trying to evaluate langkit, aiming to simplify
the development of an LSP and a set of linters/formaters for a DSL. I struggled
getting it to build, here is a lengthy summary of what I encountered and where I stopped.

The documentation recommends GNAT Community 2021.
As I was surprised that this was the latest release of GNAT, I found the blogs
mentioning alire being the recommended way to get gnat, so I gave it a try that way, on macOS.

I did a manual setup of alire for langkit and added a dependency to
langkit_support, trying to avoid having to build it.

I would have a hard time bringing the entire Ada toolchain to our devenvs,
and aimed to build as little as possible, hopefully requesting a small set to
be released (in pip using a bdist ? <3) if the eval was successful.

In the created alire.toml :

[[depends-on]]
langkit_support = "^24.0.0"

I realized that default compilation with manage.py was still trying to compile
langkit_support, I found the --no-langkit-support option to bypass it.

Suggestion : Separating langkit_support out of the dependency list might be justified given
that it's expected to be built.

Moving forward I was stumbling upon an error caused by xmlada_shared.gpr not
found from the xmlada lib. I was using the one brought in by langkit_support.

xmlada_unicode.gprxmlada_unicode.gpr:24:06: imported project file "../xmlada_shared.gpr" not found
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/.local/share/alire/builds/xmlada_24.0.0_ae5a015b/b0131ab91b91aa28704f2c86cc19d71588851835808f51b4132600c1599b3c69/unicode/xmlada_unicode.gpr"
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/.local/share/alire/builds/xmlada_24.0.0_ae5a015b/b0131ab91b91aa28704f2c86cc19d71588851835808f51b4132600c1599b3c69/distrib/xmlada.gpr"
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/.local/share/alire/builds/libgpr_24.0.0_d9c96bda/9098d495aeffac6e11d3d93c6b8c5c10b8181054d2fc132f3bf2a0f70303aa83/gpr/gpr.gpr"
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/.local/share/alire/builds/gnatcoll_24.0.0_11c512d1/47cf7d7e8a2b7b3c0e9536d64ef2252d7e51940df5bfe09d9251db5741773a95/gnatcoll.gpr"
xmlada_unicode.gpr:24:06: imported by "/Users/bourgeoisclement/repos/langkit/contrib/lkt/build/liblktlang.gpr"
xmlada_unicode.gpr:29:25: unknown package or project "Xmlada_Shared"
xmlada_unicode.gpr:29:25: wrong expression kind for attribute "library_kind"

That most likely is a problem to report to xmlada, and the way xmlada_24.0.0 is
packaged for alire.
I fixed it by running ./configure in the xmlada GPR_PROJECT_PATH location to generate the
Xmlada_Shared.gpr file. Not langkit fault I know :)

Next I ran into this :

python manage.py make --no-langkit-support

Generating source for libpythonlang...
Generating source for liblktlang...
Generation complete!
Building the generated source code
Building for config (BuildMode.prod, LibraryType.static)
gpr.gpr:26:04: value "static" is illegal for typed string "bld"

I figured out that this was caused by a conflict in the supported GPR_BUILD
values in gpr.gpr, I forced that to be "production" in the gpr file, not sure which
part of the coupling should be the source of truth, but I wanted to move
forward.

After the above, the prettier_ada dependency reported being missing, that was not mentionned in the
intro, so worth updating, I assume that it's pretty recent.
I cloned the repository, setup an alire project there as well with vss and got that solved.

Finally I blocked on :

   [Ada]          libpythonlang_support-adalog-main_support.ads
/Users/bourgeoisclement/repos/langkit/contrib/python/build/src/libpythonlang_support-adalog-solver.adb:1595:36: error: "Free" is not a visible entity of "Libpythonlang_AdaSAT"

   compilation of libpythonlang_support-adalog-main_support.ads failed

gprbuild: *** compilation phase failed
Build failed: error while running gprbuild -p -j4
-P/Users/bourgeoisclement/repos/langkit/contrib/python/build/libpythonlang.gpr
-XBUILD_MODE=prod -XLIBRARY_TYPE=static -XGPR_BUILD=static -XXMLADA_BUILD=static
-XLIBPYTHONLANG_WARNINGS=true -gnatef:

I assume that it was foolish to expect langkit_support to work on master, so I
went to 24.0 but it failed similarly.

Got this error on the release branch that did not stop the build from progressing
but still sharing it for visiblity :

Generating source for liblktlang...
/Users/bourgeoisclement/repos/langkit/contrib/lkt/language/parser.py:1248:1: Then expression should have a default value provided, in cases where the type of the provided function's return type (here Int) does not have a default null value.
Errors, exiting
Generation complete!

Retried without trying to use the langkit_support from alire, building it directly, but got a similar
error :

   [Ada]          langkit_support-adalog-solver.adb
langkit_support-adalog-solver.adb:1595:22: error: "Free" is not a visible entity of "AdaSAT"

   compilation of langkit_support-adalog-solver.adb failed

Should I use an older gnat ? I was using 13.2.0.

Final note, manage.py seems to run twice, always retrying after a failure, not a big deal but that was surprising.

Let me know some of my notes are unclear or if you need more context.

[raph-amiard]

Hi Clement, thanks for opening this issue! Glad to hear from you aswell 🙂

We're going to get to each issue in due time, first

The documentation recommends GNAT Community 2021.
As I was surprised that this was the latest release of GNAT, I found the blogs
mentioning alire being the recommended way to get gnat, so I gave it a try that way, on macOS.

I'm surprised, I'm not finding this reference, could you point me to it ?

We in any case need to update the build instructions. We're going to do a first pass trying to use Langkit on a recent Ubuntu.

However, for the moment, nobody in the team has access to a macOS machine. I propose that we already do a pass on our side and ping you when that is done so you can see if things are already better.

Thanks!

[cbourgeois]

Happy to help incrementally !

This first reference is this link that lands on Adacore download page for GNAT community 2021 :
https://github.com/AdaCore/langkit/blob/master/README.md?plain=1#L25

[pmderodat]

Hello @cbourgeois!

We discussed this internally, but realized that the “full Alire way” is probably a dead end: as you noticed, using a mix of Alire releases and master branches is bound to cause problems: all master branches more or less form a rolling release. As a consequence, except for the compiler, it is best to use the same version for all libraries (xmlada, libgpr, gnatcoll, adasat, prettier, langkit). As we already discussed, you’ll probably be better off with Langkit’s master branch, so your only option will be to build all dependencies yourself: Alire will help only to give you a working toolchain. I’ll update the README accordingly.

In the future, we could envision creating a binary distribution for Langkit similar to what you described during our live discussions, shipping GNAT, pre-built dependencies and possibly Python+Java distributions to use the full ecosystem that we are currently building. But we’re not there yet.

Should I use an older gnat ? I was using 13.2.0.

13.2.0 should be completely fine. :-) It seems that all the problems you reported in your opening message are related to the use of incompatible versions for dependency libraries: please let us know if you need help building one of them, or if you still have compilation errors with master-only deps.

Final note, manage.py seems to run twice, always retrying after a failure, not a big deal but that was surprising.

That’s unexpected. What are the symptoms exactly? Note that manage.py compiles both liblktlang and libpythonlang in parallel, so it’s expected that you have one failing and one still running at the same time.

[simonjwright]

Not sure if it's related, but for some time there have been problems with macOS vs GPRBuild trying to create 'static Stand Alone Libraries (SAL)'; I found a reasonable patch for langkit v25.0.0:

diff --git a/langkit/templates/project_file.mako b/langkit/templates/project_file.mako
index c3786b1b8..3fb1892de 100644
--- a/langkit/templates/project_file.mako
+++ b/langkit/templates/project_file.mako
@@ -45,6 +45,15 @@ library project ${lib_name} is
    Library_Standalone : Library_Standalone_Type := external
      ("${upper_lib_name}_STANDALONE", external ("STANDALONE", "standard"));
 
+   --  For macOS: static SALs can be generated but can't be linked, at
+   --  least with the technique used by gprbuild.
+   case Library_Kind_Param is
+      when "relocatable" =>
+         null;
+      when "static" | "static-pic" =>
+         Library_Standalone := "no";
+   end case;
+
    type Boolean is ("false", "true");
 
    Externally_Built : Boolean :=

My current script for building langkit is here. I suspect you wouldn't want to say --disable-all-mains.

[saper]

@simonjwright I noticed you add -fPIC - isn't it the purpose of the static-pic and relocatable profiles to add -fPIC and static should not have it?