running on nixos is generating transaction.go:6:5: undefined: conn

[ymolists]

Hi folks

I am trying to run rill on nixos. nisxos does not allow using embedded binaries . All binaries mus be compiled with the same version and rt as nixos. so my guess is you are trying to embed a previously build duckdb maybe ?

Here is the error when i try to compile the latest from the main branch. Can someone please help me with some hints on what could be happening.

                              861.57 kB
.svelte-kit/output/server/entries/pages/(application)/(workspace)/files/_...file_/_page.svelte.js                                      902.96 kB
✓ built in 49.97s

Run npm run preview to preview your production build locally.

> Using @sveltejs/adapter-static
  Wrote site to "build"
  ✔ done
rm -rf cli/pkg/web/embed/dist || true
mkdir -p cli/pkg/web/embed/dist
cp -r web-local/build/* cli/pkg/web/embed/dist
rm -rf runtime/pkg/examples/embed/dist || true
mkdir -p runtime/pkg/examples/embed/dist
git clone --quiet https://github.com/rilldata/rill-examples.git runtime/pkg/examples/embed/dist
rm -rf runtime/pkg/examples/embed/dist/.git
cp scripts/install.sh cli/pkg/installscript/embed/install.sh
go run scripts/embed_duckdb_ext/main.go
# github.com/marcboeker/go-duckdb
/home/ymo/go/pkg/mod/github.com/marcboeker/go-duckdb@v1.8.3/transaction.go:6:5: undefined: conn
make: *** [Makefile:26: cli.prepare] Error 1

same happens when i try to compile as well

❯ go build -o rill cli/main.go 

# github.com/rilldata/rill/runtime/pkg/activity
runtime/pkg/activity/sink_kafka.go:39:18: undefined: kafka.Producer
runtime/pkg/activity/sink_kafka.go:42:22: undefined: kafka.LogEvent
runtime/pkg/activity/sink_kafka.go:48:29: undefined: kafka.LogEvent
runtime/pkg/activity/sink_kafka.go:49:25: undefined: kafka.NewProducer
runtime/pkg/activity/sink_kafka.go:49:44: undefined: kafka.ConfigMap
runtime/pkg/activity/sink_kafka.go:90:36: undefined: kafka.Message
runtime/pkg/activity/sink_kafka.go:91:26: undefined: kafka.TopicPartition
runtime/pkg/activity/sink_kafka.go:91:75: undefined: kafka.PartitionAny
runtime/pkg/activity/sink_kafka.go:97:21: undefined: kafka.Error
runtime/pkg/activity/sink_kafka.go:164:54: undefined: kafka.LogEvent
runtime/pkg/activity/sink_kafka.go:97:21: too many errors
# github.com/marcboeker/go-duckdb
/home/ymo/go/pkg/mod/github.com/marcboeker/go-duckdb@v1.8.3/transaction.go:6:5: undefined: conn

Best Regards

[begelundmuller]

so my guess is you are trying to embed a previously build duckdb maybe?

Hi @ymolists! Yes we do indirectly, and the same happens for Kafka. That could be the problem you are running into here (though it may also be a problem with CGO generally, not sure how that works in NixOS).

To be specific, as far as I am aware we have two dependencies that use pre-built binaries:

• https://github.com/marcboeker/go-duckdb for DuckDB, which by default uses pre-built static libraries. If that doesn't work for you, its README describes how you can override it to use a dynamic library instead (which you can build/manage yourself).
• https://github.com/confluentinc/confluent-kafka-go for Kafka, which also uses pre-built static libraries for librdkafka by default. Its README similarly describes how you can override it to use a dynamic library.

If you manage to get Rill working with NixOS, I would love to hear about the steps it takes. It would also be a useful reference for other NixOS users!