Skip to content

Odoo 18: custom DB name, demo data choice & OCA localizations at firstboot - #26

Open
marcos-mendez wants to merge 2 commits into
turnkeylinux-apps:masterfrom
popsolutions:dev2
Open

Odoo 18: custom DB name, demo data choice & OCA localizations at firstboot#26
marcos-mendez wants to merge 2 commits into
turnkeylinux-apps:masterfrom
popsolutions:dev2

Conversation

@marcos-mendez

@marcos-mendez marcos-mendez commented Feb 5, 2024

Copy link
Copy Markdown

Odoo 18: custom DB name, demo data choice & OCA localizations at firstboot

Status: open again to continue the work — not a merge request yet.
The appliance build is the missing piece: this has not been built or
boot-tested on tkldev. Code, logic and syntax are verified and the unit
tests pass, but until someone runs a real build this should be treated
as work in progress. Reviews and build testing are exactly what it needs.

This PR has been completely rebuilt — as promised in my 2026-05-14 comment. The old Odoo 16 implementation is gone; the branch is now #31 (Odoo 18 upgrade) plus a clean rewrite of the firstboot features this PR originally attempted. The [unstable] marker is dropped.

If #31 is merged first, this PR reduces to a single feature commit.

The original blocker is fixed

@JedMeister's first review asked for non-interactive pre-seeded launch (TurnKey Hub). Now: when a password is pre-seeded, no dialog is ever shown and unset options fall back to safe defaults, so existing Hub deployments keep working with APP_PASS alone. Everything below is opt-in:

export APP_PASS=SecretPassword1!
export ODOO_DB_NAME=production   # default: keep the configured database
export ODOO_DEMO=no              # default: no demo data
export ODOO_L10N=brazil,spain    # default: none

Interactive turnkey-init asks via the standard Dialog wrapper (proper yesno/inputbox, no more "type yes/no" text prompts).

What it adds

1. Custom database name — new databases are initialized properly via odoo -i base --stop-after-init. The old code did a bare CREATE DATABASE, leaving an uninitialized DB — the "database creation is not triggered" bug I reported here in 2024.

2. Demo data choice--without-demo=all unless demo data is explicitly requested, so production is the safe default. Only applies when a new database is initialized.

3. OCA localization addons — selected l10n-* repos are cloned from the pinned 18.0 branch into /opt/odoo/custom-addons, together with their oca_dependencies.txt chain; python requirements installed, addons_path updated. Nothing is auto-installed into the database — the modules simply show up in the Apps menu. The whole step is best-effort: no network, no problem, firstboot still completes.

Data safety

Re-running turnkey-init is a documented flow, so it must never surprise you:

  • The database name defaults to the one configured in odoo.conf, not to the example DB — a custom database can't be orphaned by a re-run.
  • TurnkeylinuxExample is dropped only after a replacement database has been successfully initialized. No other database is ever dropped.
  • An existing database that isn't an Odoo database is refused, not adopted.
  • If initialization fails, the partial database is removed and odoo is restarted before exiting non-zero — the appliance is never left with Odoo down or a half-initialized DB.

Security

The previous version of this PR connected to PostgreSQL as postgres over TCP using the Odoo admin password — which never worked (the appliance uses peer auth). All database operations now go through runuser -u postgres / runuser -u odoo. Also:

  • Master password stored hashed (pbkdf2_sha512) in odoo.conf instead of plaintext.
  • Password passed via ODOO_PASS env rather than argv (/proc/<pid>/cmdline is world readable) and removed from the environment before any subprocess runs.
  • odoo.conf is chmod 0640 — it holds the db password and the master password hash.
  • Database names validated against ^[A-Za-z][A-Za-z0-9_-]*$ before reaching SQL; admin matched by login instead of a hardcoded id=2.
  • Repo names coming from third-party oca_dependencies.txt are validated, clone destinations are confined to the addons dir, git clone gets -- and runs as the odoo user, and the dependency chain is capped.

Tests

tests/test_inithook.py — 18 tests, runs anywhere, no appliance needed:

$ python3 tests/test_inithook.py
Ran 18 tests in 0.016s

OK

It stubs postgres/dialog/odoo and covers the pre-seeded and interactive paths, the create/adopt/refuse decisions, failure recovery, password non-leakage, and rejection of hostile oca_dependencies.txt entries. tests/ is outside overlay/, so nothing ships in the appliance image.

Still needs — the build

The appliance build is the one thing that has not been done. No tkldev
build, no ISO, no boot test of the resulting appliance. What has been
verified is the code itself: syntax, control flow, and the 18 unit tests
above, all of which run without an appliance.

So concretely, still open:

  • tkldev build of the appliance from this branch
  • firstboot on the built image: pre-seeded (inithooks.conf) and interactive
  • one run with ODOO_L10N set, to confirm the OCA clone path on real hardware

I'd rather have this open and visible while that happens than sitting in draft
for another year. Happy to do the build myself once #31 lands — and if anyone
has a tkldev box handy sooner, testing is very welcome.

Related

@JedMeister

Copy link
Copy Markdown
Member

Thanks for this @marcos-mendez - From a glance it mostly looks good, but I'll need to tweak it a bit to handle non-interactive pre-seeded launch (e.g. from the TurnKey Hub). I'll try to get that done this week so I can build the updated Odoo.

@marcos-mendez

marcos-mendez commented Feb 7, 2024 via email

Copy link
Copy Markdown
Author

@JedMeister

JedMeister commented Feb 7, 2024

Copy link
Copy Markdown
Member

Hi @marcos-mendez,

I need an OpenSource Alternative do TKLBAM for working in the
migration of Odoo to newer version an tkl-odoo versions.

FWIW TKLBAM IS open source - although using it without the Hub requires you to generate your own profile. Having said that, all the pieces are there and I'd be happy to assist you to put it together.

I got stucked on Odoo 14 and de migration process to a 16v is not that
trivial. HAving to tweak a lot of things.

Having said the above (re TKLBAM), even with a custom profile, I still don't think it's going to give you what you are after. Without either pre-scripting an Odoo14 -> Odoo16 migration and/or manually migrating, it won't just automagically migrate your data to a newer version of Odoo. 😞

Also Python3.7 dependent software got just out offline as this is EOL for
python3.7 and im having to migrate to docker if i don't manage to do this
migration my self to another LXC/TKL project

If you're using TurnKey v16.x (based on Debian 10/Buster) currently - then the core OS (inc python 3.7) is covered by the Debian LTS team until mid year. So whilst Python upstream is EOL, it's still supported in Debian. AFAIU the bigger issue would be support for Odoo v14.x itself wouldn't it?

Perhaps I'm missing something, but I'm also not sure how a Docker container would help here? Whether you use TurnKey or a Docker container, you'd still need to migrate the DB and custom files, etc wouldn't you? And you'd still have Odoo14 - which as I understand it is no longer supported by upstream?!

FWIW the v17.1 TKL appliance includes Odoo 14.x - installed from Debian repos (so some paths may be different - but I wouldn't imagine that it would be too hard?). FYI, there also seems to be another version in bullseye-backports - but I'm not clear on what advantages/improvements may be included as it appears to be the same upstream version of Odoo?! If the version in main works, I'd be inclined to stick with that.

That includes Odoo v14 too, but AFAIK it's supported by the Debian security team, until August 2024. That's when Debian 11/Bullseye will be moved to LTS and whilst the Odoo package will remain in the repos (possibly not the backport one though?) - support may end there? The LTS team have limited resources, so for a package like Odoo, I suspect that you/your client would want to subscribe to an LTS maintainer (i.e. pay a monthly subscription)- or assist maintaining it yourself if you wanted/needed ongoing support of Odoo v14.x.

But perhaps the extra 6 months of support gives you enough time to migrate the server to Odoo v16.x?

FYI the v17.1 appliance includes Python 3.9 so any custom code you've written may require some minor tweaks - although I imagine it should mostly "just work" (IIRC there wasn't that much difference between python v3.7 & v3.9). As noted above, python is a core system component so I'm sure it would be maintained by the LTS team for the life of the LTS (so until circa mid 2026).

Do you think is an easy to update from buster to bullseye without the pain
of migrating databases manually?

That should be possible. Although like I say, IIRC the Odoo installation itself will be slightly different (Official Debian package using files in /usr/lib etc vs upstream package probably all in /opt). For the OS and most packages - see the Debian docs. Note those docs are fairly exhaustive and personally there's lots of bits I personally skim over. But I only do that with desktop Linux systems; for a production server - especially for a hospital - I would probably do it all as close to the Debian docs as practical.

Personally, whilst it can be a little more involved, I much prefer to migrate data to a new clean server. It's a little more work, but gives you a better, cleaner end result IMO. OTOH a Debian "in place" upgrade is a completely acceptable path.

Note that the TurnKey apt repo will need the updated keys added manually (as well as updating the sources.list & security.sources.list from buster to bullseye). It's pretty easy though and obviously I can assist there.

My experience suggests that a Debian upgrade may be relatively pain free - but I've also had painful experiences, so I'm not completely sure...

Also do you think it can be done in a
script like debian best practices one?

TBH, I'm not 100% sure what you mean by that? I'm not aware of any "best practices" upgrade script that would handle this sort of upgrade?! And even if there was, I'd personally be very wary and wouldn't trust it unless I'd already run it successfully on a clone of the system in question.

Personally I would do it manually - ideally first on a snapshot of your current server first. I would also recommend recording the process and take notes of any custom steps/changes and/or issues you encounter along the way.

If it's installed to bare metal, use a tool such as Clonezilla to image the disk and create a VM from the disk image for the initial "trial" upgrade. Then once you're happy, rerun the same steps on the production system. Alternatively, if it's not installed on bare metal, then you could do it in a separate VM (started from a clone of the prod server) first, and if all goes well, just switch to the new server. If it all goes smoothly, that will save some work - and mean minimal downtime. Either way - if it doesn't go smoothly at least the prod server remains unaffected.

I'm happy to assist where I can, but all things considered TBH I don't see a guaranteed easy path for you. Sorry to be the bearer of bad news. 😢

Having said all that, please feel free to prove me wrong! 😁 I'd rather give you the brutal truth as I see it and have you surprise me, than the opposite...

Take care man and good luck with it all. Hopefully I'll have this new updated appliance ready by early next week, but no promises.

@marcos-mendez marcos-mendez changed the title Odoo OCA Installation Odoo 16.0 + OCA Wrapper [18.0 TKLDEV-bookworm][unstable] Mar 31, 2024
@marcos-mendez

Copy link
Copy Markdown
Author

Discovered some issues with custom database build for OCA. Still marking this as unstable.

The databasecreation is not triggered automatically as it should. And the odoo.conf gets overwriten by the turnkey-updates.

@marcos-mendez

Copy link
Copy Markdown
Author

@JedMeister can you push the 18.0 default main image to the tkl repo? I think it will take a longer work to me and the current 18.0 looks like ready to ship as it is. No reason to my work delay the release of the app

Sorry for that man.

@marcos-mendez
marcos-mendez marked this pull request as draft April 1, 2024 00:46
@JedMeister

Copy link
Copy Markdown
Member

Hi Marcos, no worries I'll do that ASAP. It might take a little while for us to publish it as we do the releases in batches of 10 (or more if there are more than 10 ready).

But I'll ensure that it makes it into the next batch.

Take care man and hopefully speak more soon.

@l-arnold

Copy link
Copy Markdown

Listening here...

Major changes:
- Odoo 18 from official nightly apt repo (was v16 from backports)
- Added WebSocket proxy support for live chat/notifications
- Updated odoo.conf with production-optimized defaults
- Added /opt/odoo/custom-addons for OCA/custom modules
- Updated inithook for Odoo 18 password hashing
- Installed rtlcss and wkhtmltopdf 0.12.6
- Updated README and changelog
@l-arnold

Copy link
Copy Markdown

Thanks for this work @marcos-mendez — I haven't tested yet but have reviewed the code carefully across both PRs. Should we discuss how we bring the best of both (Odoo 16 and Odoo 18 targeted) forward as we flesh out the tkl19 build? See the related discussion at turnkeylinux/tracker#2111.
Would you like me to do any testing on this in the meantime?

(My same essential comment just also went to the odoo 18 focused PR)

@marcos-mendez

Copy link
Copy Markdown
Author

Please don't consider this, i rather prefer to make this a total rebuild. Now a days i changed the workflow complitly from this approach @l-arnold Just working with OCB wil bee fine. The date are already squeezed to the releaase of 19 i prefer to take a step backward now

Rebuild of this PR's features on top of the Odoo 18 base, replacing the
Odoo 16 implementation. The inithook is rewritten around three opt-in
options, all of which also work pre-seeded (TurnKey Hub):

  --dbname / ODOO_DB_NAME   database name (default: keep configured one)
  --demo   / ODOO_DEMO      demo data for a new database (default: no)
  --l10n   / ODOO_L10N      OCA localization addons to download

Behaviour:

* New databases are initialized via 'odoo -i base --stop-after-init'
  (with or without demo data) instead of a bare CREATE DATABASE, which
  used to leave an uninitialized database behind.
* The example database is only dropped once a custom database has been
  successfully initialized to replace it; no other database is ever
  dropped, and an existing database that is not an Odoo database is
  refused rather than adopted.
* Re-running turnkey-init defaults to the database configured in
  odoo.conf, so a custom database is never orphaned.
* If initialization fails, the partial database is removed and odoo is
  restarted before exiting non-zero.
* OCA l10n-* repos are cloned (as the odoo user) from the pinned 18.0
  branch into /opt/odoo/custom-addons together with their
  oca_dependencies chain, python requirements are installed and
  addons_path is updated. Repo names are validated and clone
  destinations are confined to the addons dir; the whole step is
  best-effort and never fatal to firstboot.

Non-interactive support (the original review blocker): when a password
is pre-seeded no dialog is ever shown and unset options fall back to
safe defaults, so existing Hub deployments keep working with APP_PASS
alone. Interactive runs use the standard Dialog wrapper.

Security:

* master password stored hashed (pbkdf2_sha512) instead of plaintext
* password passed via ODOO_PASS, not argv (/proc/PID/cmdline is world
  readable), and removed from the environment before any subprocess
* odoo.conf chmod 0640 - it holds the db password and master pw hash
* database names validated before reaching SQL; 'admin' user matched by
  login rather than the hardcoded id=2

Adds tests/test_inithook.py: 18 tests covering the pre-seeded and
interactive paths, database decisions, failure recovery and rejection
of hostile oca_dependencies.txt entries. Runs without an appliance.
@marcos-mendez marcos-mendez changed the title Odoo 16.0 + OCA Wrapper [18.0 TKLDEV-bookworm][unstable] Odoo 18: custom DB name, demo data choice & OCA localizations at firstboot Aug 12, 2026
@marcos-mendez
marcos-mendez marked this pull request as ready for review August 12, 2026 04:56
@marcos-mendez

Copy link
Copy Markdown
Author

Rebuilt from scratch and taken out of draft — the Odoo 16 implementation is gone, this is now Odoo 18 (#31) plus a clean rewrite of the firstboot features.

@JedMeister the non-interactive/pre-seeded requirement from your very first review is finally honoured: with APP_PASS set no dialog is ever shown and every new option falls back to a safe default, so Hub launches behave exactly as they do today. ODOO_DB_NAME, ODOO_DEMO and ODOO_L10N are optional pre-seeds on top.

The 2024 bug I reported here ("database creation is not triggered") was a bare CREATE DATABASE with no odoo -i base — fixed. The old psycopg2-as-postgres-over-TCP approach is gone too; it could never have worked against peer auth.

Added tests/test_inithook.py (18 tests, no appliance required) so the risky paths are demonstrable rather than asserted — including that re-running turnkey-init can't orphan a custom database, and that a failed init doesn't leave Odoo down.

@l-arnold this should be the consolidation you proposed for tkl19 (turnkeylinux/tracker#2111). Testing very welcome — build testing on tkldev is the main thing I haven't done.

@marcos-mendez

Copy link
Copy Markdown
Author

Status note: this PR is open again to continue the job, not because it is finished.

To be explicit about what is missing: the build. Nothing here has been built or boot-tested on tkldev. No ISO, no appliance image, no firstboot run on real hardware. What is verified is the code — syntax, control flow, and 18 unit tests (tests/test_inithook.py) that run without an appliance and cover the pre-seeded path, the interactive path, the database create/adopt/refuse decisions, failure recovery, and rejection of hostile oca_dependencies.txt entries.

Still to do before this is merge-ready:

  • tkldev build of the appliance from this branch
  • firstboot on the built image, both pre-seeded via inithooks.conf and interactive
  • one run with ODOO_L10N set, to confirm the OCA clone path works outside the test stubs

I took it out of draft deliberately: it sat in draft for two years and nobody could review it there. Open and honest about its state beats invisible. Please read it as work in progress — review comments now are more useful to me than a merge.

I'll do the build myself once #31 lands. @l-arnold if you still have capacity to test, this is the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants