Proposal: Metacli for Metabuild contents information fetching

[bhavyaVeera]

[Proposal]: Using meta_cli for fetching flashable binaries, programmers, raw and patch XMLs from meta build.

The target meta_builds are providing meta_cli binary to fetch the images required for flashing. This has been in use in PCAT since the time of past discussions with target teams for maintaining consistent fetching logic.

Sample location: \sundae\scratch-nsid-sd-05\Kaanapali.LA.2.0-00043-STD.INT-2\common\build\app\windows_x86_64\meta_cli.exe

This PR is to modify the below logical flow in QDL. It first tries to fetch through the meta_cli approach and if it fails for some reason, it falls back to the existing logic of parsing contents XML directly.
Introducing metacli_locate(), metacli_contents_decode_selectors() and populate_from_metacli()
This also introduces a new selector sku for targets like Nord

I'd appreciate your suggestions, comments and feedback to make this PR better.

contents_load():
    if metacli_locate():
        # Resolve storage/flavor/sku against meta_cli itself, since no
        # entries exist yet to validate against the usual way.
        selectors = metacli_contents_decode_selectors(pattern_copy)
        if ok: populate_from_metacli(selectors[0])
        if both ok: populated = true
        else: free selectors, reset contents to fall back to XML

    if not populated:
        load_xml()
        selectors = decode_selectors(pattern)   # validates against entries

    find_programmers(selectors[0])              

    for (storage, flavor) in selectors:          # supports comma-separated multi-target
        configure(storage)
        load matching PROGRAM + PATCH entries

    cleanup, return	

Sample Run runs the below commands

15:10:53.786 Running meta_cli (binary): \\crmhyd\nsid-hyd-06\Hawi.LA.1.0-01053-STD.INT-1\common\build\app\windows_x86_64\meta_cli.exe  arg[1]: get_storage_typesmeta_cli stdout (15 bytes):

15:10:55.743 Running meta_cli (binary): \\crmhyd\nsid-hyd-06\Hawi.LA.1.0-01053-STD.INT-1\common\build\app\windows_x86_64\meta_cli.exe  arg[1]: get_product_flavorsmeta_cli stdout (16 bytes):

15:10:56.144 Running meta_cli (binary): \\crmhyd\nsid-hyd-06\Hawi.LA.1.0-01053-STD.INT-1\common\build\app\windows_x86_64\meta_cli.exe  arg[1]: get_partition_files  arg[2]: group=True  arg[3]: flavor=asic  arg[4]: storage=ufs  arg[5]: critical=Falsemeta_cli stdout (14365 bytes):

15:10:56.859 Running meta_cli (binary): \\crmhyd\nsid-hyd-06\Hawi.LA.1.0-01053-STD.INT-1\common\build\app\windows_x86_64\meta_cli.exe  arg[1]: get_device_programmer_config  arg[2]: storage=ufs  arg[3]: flavor=asicmeta_cli stdout (1611 bytes):

[JohnSagaQuic]

One thing to keep in mind is we don't have meta-cli on Mac platform at this point.

[JohnSagaQuic]

Try to move all platform-specific code into oscompat.h/.c so the rest of the codebase remains clean and independent of OS-specific differences. For MacOS, since currently meta_cli won't work, the function should return no, so it will fall back to existing logic.

[JohnSagaQuic]

I assumed the meta_cli is not working for MacOS yet, at least no one have verified that it's working.

[igoropaniuk]

Hi @bhavyaVeera ,

Thanks for the proposal and the detailed write-up; it makes the intent easy to follow. I'm not in favour of this approach, though, and I'd like to explain why before we go further on the implementation.

The core concern: qdl should not wrap other tools

The PR makes qdl search the build tree for meta_cli/meta_cli.py and execute it implicitly, so qdl becomes a front-end for another tool. I've been through this before with the DTE tool. It started as an assembler/disassembler for DTB containers and gradually grew wrappers for flashing, signing and so on. Each wrapper looked reasonable on its own, but together they caused several problems:

• Coupling to another tool's lifecycle. The wrapped tool's CLI, output format and behaviour change on its own schedule, owned by another team. Every change becomes a qdl bug, and there's no stable, documented contract to code against.

• Unclear ownership when things break. If a flash fails, is it qdl, meta_cli, the Python on the host, or the build tree? Users file the bug against the tool they ran, which here would be qdl.
• Untestable upstream. meta_cli ships only inside internal meta builds (the paths in the description are internal shares), so upstream CI and external users can't exercise or reproduce this path at all.

What I'd suggest instead (multiple options):

• Document the workflow, don't hard-wire it. Add a section to the README (or a docs/ guide) describing how to flash from a meta build: run meta_cli to resolve the storage, flavor and SKU and collect the programmer, rawprogram and patch files; then invoke qdl with them, either the classic way (qdl --storage ufs <programmer> rawprogram*.xml patch*.xml) or through qdl flash. A small script can chain the two steps; that orchestration belongs in the caller, not inside qdl.
• Use qdl's existing inputs as the contract. qdl already understands contents.xml and the flashmap format (qdl flash <flashmap>). If meta_cli could emit a qdl flashmap, or a directory qdl can consume directly, the hand-off becomes a documented, versionable file instead of a subprocess call.
• Fix gaps in qdl's own parsing directly. If contents.xml support is missing something, such as the new SKU selector for targets like Nord, that's a welcome, self-contained PR to qdl's contents parser. It needs no external tool, and we can test it upstream with sample XMLs.

Thanks again for raising this; consistent fetching logic is a real problem, and I think it's best solved next to qdl rather than inside it.

@andersson I would like your opinion on this as well