feat: add registry command to manage global and local skill registries, drop built-in registries

[jakemac53]

This task ended up snowballing a lot and includes various registry related improvements and a new set of commands:

• Adds registry add, registry list, and registry remove commands.
  • These support both git URIs and owner/repo formats (which infers GitHub), as trailing arguments. Multiple registries can be added/removed at time.
• Adds a global config file to store permanent configuration for a machine.
• Adds support for globally installed registries as well as project level registries.
  • Includes tracking back links to where registry skills were installed so that when you remove a global registry it can go clean up all of those skills.
• Adds support for running global migrations, and adds a migration for existing projects. It will ask if they want to retain the existing registries or not. This supports a graceful migration for existing users.
• The API can still just provide its own list of registries, so serverpod can automatically provide registries if it wants. But the core package does not have any built in registries.
• Registry layout (flat vs nested) is now inferred and not hard coded in the config.

Closes dart-lang/ai#463

Note that this PR and #14 are stepping on each toes a bit, whichever lands first though I can reconcile the other one. Or possibly we should just not do #14, as deleting .dart_tool will end up also effectively removing any locally installed registries as well, I am a bit unsure about that.

[jakemac53]

cc @vlidholt I have a few prs out if you could help me get reviews :)

[marcelomendoncasoares]

Note that this PR and #14 are stepping on each toes a bit, whichever lands first though I can reconcile the other one. Or possibly we should just not do #14, as deleting .dart_tool will end up also effectively removing any locally installed registries as well, I am a bit unsure about that.

Hi @jakemac53! I just approved and merged your other #14 PR, but only afterwards I saw this note. Indeed, losing registries on a flutter clean would not be a good UX. However, do we need registries to be managed imperatively? How about using a skills key on the workspace or package pubspec.yaml? Then a next skills get will read the declared registries and sync.

Also, whenever you need an opinion or review, just mark me on a comment or for review and I'll try to contribute. But feel free to merge other smaller/simple changes directly.

[jakemac53]

How about using a skills key on the workspace or package pubspec.yaml? Then a next skills get will read the declared registries and sync.

I don't think the pubspec is a good place because in many cases you wouldn't want these settings to be persistent across all users (maybe some developers of the package want certain skills but others don't etc).

There will also be a lot more configuration and data stored here in the future - specific information about exactly which skills were installed, at what versions, etc etc. That information is almost certainly specific to an individual user.

Also, whenever you need an opinion or review, just mark me on a comment or for review and I'll try to contribute. But feel free to merge other smaller/simple changes directly.

Ok, I do generally want reviews but I could add a googler for those reviews instead for some of these

[jakemac53]

Another idea I had to resolve this is to basically have a "recovery" mode based on metadata injected into each installed skill directory. So if this package installs a skill it dumps a file in there saying where it came from, also some of the info such as the version or hash of its contents. Some of that data may only exist in the skill install dir.

This way we could recover the config as long as any skill was in fact installed from a registry.

[jakemac53]

cc @kenzieschmoll did you want to look at this?

I am curious what your thoughts are in particular around the local configuration location.

[jakemac53]

Ugh, now the file name in the test is 2 characters too long for windows. Fun. worked around this with some git foo

[kenzieschmoll]

should there be an easy way to remove all?

[jakemac53]

I don't think its as necessary for this one, you won't have nearly as many registries as skills.

[kenzieschmoll]

would a git uri pass this check?

[jakemac53]

This whole code path is only meant to handle the owner/repo format, which we special case for GitHub repos.

Anything with a : or an @ (http or ssh URIs) will fall back to the simpler case where we just use the URI directly.