Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/contents.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,8 @@ operator, user, and contributor references are easier to find.
- [v0-1-0-migration-guide.md](v0-1-0-migration-guide.md): Migration notes for
the v0.1.0 child-environment API, glob behaviour, and serial-dependency
additions, plus the stability caveat that covers them.
- [v0-1-1-migration-guide.md](v0-1-1-migration-guide.md): Migration note for
replacing no-op aggregate recipes with dependency-only actions or targets.
- [users-guide.md](users-guide.md): End-user reference for authoring and
running Netsuke manifests, including executable discovery and
`command_available` branch selection.
Expand Down
188 changes: 102 additions & 86 deletions docs/developers-guide.md

Large diffs are not rendered by default.

458 changes: 373 additions & 85 deletions docs/netsuke-design.md

Large diffs are not rendered by default.

15 changes: 12 additions & 3 deletions docs/users-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,10 @@ offending key.

### Rules and recipes

A rule or target must provide exactly one recipe:
Rules must provide exactly one executable recipe. Actions and targets that
perform work must also provide exactly one recipe, but may omit it when a
non-empty `deps` list is their complete operation. This dependency-only
aggregate form is preferred over a no-op command such as `command: ":"`:

- `command`: one shell command, or an ordered list of commands.
- `script`: a multi-line POSIX shell script.
Expand Down Expand Up @@ -376,7 +379,10 @@ platform-selected actions when a manifest must work on Windows.
A target supports these fields:

- `name`: one output path or a list of output paths.
- `rule`, `command`, or `script`: exactly one recipe.
- `rule`, `command`, or `script`: exactly one recipe for work that has its own
execution step. An action or target with a non-empty `deps` list may omit a
recipe to form a dependency-only aggregate; this is preferred over a no-op
`command: ":"`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- `sources`: explicit inputs. They affect freshness and become `{{ ins }}`.
- `deps`: implicit dependencies. They affect freshness but do not become
recipe arguments. Declare them on each target; reusable rules reject `deps`.
Expand Down Expand Up @@ -426,7 +432,6 @@ actions:
- name: test
command: "echo testing"
- name: all
command: ":"
dependency_order: serial
deps:
- check-fmt
Expand All @@ -445,6 +450,10 @@ targets:
- release-notes
```

The `all` action has no recipe because its dependencies are the complete
workflow. Netsuke lowers it to a native Ninja `phony` node, so it has no shell
no-op of its own.

For a serial list, Netsuke starts each direct dependency only after the
preceding one succeeds. If an earlier dependency fails, later dependencies in
that list do not start through the serial path. Repeated or shared dependencies
Expand Down
33 changes: 33 additions & 0 deletions docs/v0-1-1-migration-guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Migrating to v0.1.1

Netsuke v0.1.1 keeps every v0.1.0 manifest compatible and removes one
declarative-manifest paper cut: an action or target whose dependencies are its
entire operation no longer needs a shell no-op recipe.

## Replace no-op aggregate recipes

When an aggregate action exists only to group or order dependencies, remove
`command: ":"`. Leave its non-empty `deps` list and any
`dependency_order: serial` policy unchanged:

```yaml
actions:
- name: all
dependency_order: serial
deps:
- check-fmt
- lint
- test
```

Netsuke lowers this entry to a native Ninja `phony` node. The dependencies
retain their previous ordering, deduplication, and failure-propagation
behaviour, but the aggregate no longer launches a shell command.

An entry with neither a recipe nor a non-empty `deps` list remains invalid.
Continue using `command`, `script`, or `rule` whenever the action or target has
work of its own to perform. See the
[users' guide](users-guide.md#targets-inputs-and-dependencies) for the manifest
contract and
[serial dependency ordering](users-guide.md#run-direct-dependencies-serially)
for ordered aggregates.
53 changes: 46 additions & 7 deletions src/ast/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -147,16 +147,19 @@ pub struct Rule {

/// Execution style for rules and targets.
///
/// Exactly one variant must be provided for a rule or target. The fields are
/// flattened in the manifest, so the presence of `command`, `script`, or `rule`
/// determines the variant.
/// Rules require an executable variant. Targets and actions whose non-empty
/// `deps` list is the complete operation use an empty internal command marker.
/// The fields are flattened in the manifest, so the presence of `command`,
/// `script`, or `rule` determines an executable variant.
#[derive(Debug, Clone, PartialEq, Serialize)]
pub enum Recipe {
/// A shell command, given as a scalar or an ordered list executed by a
/// fail-fast shell chain.
Command {
/// A scalar command passes through unchanged; list entries are
/// evaluated in brace groups joined by a fail-fast `&&` chain.
/// evaluated in brace groups joined by a fail-fast `&&` chain. An
/// empty value is reserved for dependency-only manifest entries.
#[serde(skip_serializing_if = "StringOrList::is_empty_marker")]
command: StringOrList,
},
/// An embedded multi-line script.
Expand All @@ -171,6 +174,17 @@ pub enum Recipe {
},
}

/// Preserve the established diagnostic for entries without a recipe or deps.
pub(crate) const MISSING_RECIPE_ERROR: &str = "missing one of command, script, or rule";

impl Recipe {
/// Report whether this recipe lowers to a dependency-only Ninja node.
#[must_use]
pub(crate) const fn is_dependency_only(&self) -> bool {
matches!(self, Self::Command { command } if command.is_empty_marker())
}
}

/// Flattened recipe fields before deserialization selects a variant.
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
Expand Down Expand Up @@ -205,9 +219,9 @@ impl<'de> Deserialize<'de> for Recipe {
},
(None, Some(script), None) => Ok(Self::Script { script }),
(None, None, Some(rule)) => Ok(Self::Rule { rule }),
(None, None, None) => Err(serde::de::Error::custom(
"missing one of command, script, or rule",
)),
(None, None, None) => Ok(Self::Command {
command: StringOrList::Empty,
}),
(command_opt, script_opt, rule_opt) => {
let present: Vec<&str> = [
("command", command_opt.is_some()),
Expand All @@ -226,3 +240,28 @@ impl<'de> Deserialize<'de> for Recipe {
}
}
}

impl NetsukeManifest {
/// Validate that every dependency-only entry has declared dependencies.
///
/// A rule cannot be dependency-only because targets need a reusable
/// executable recipe when they reference a rule by name.
pub(crate) fn validate_recipes(&self) -> Result<(), &'static str> {
if self
.rules
.iter()
.any(|rule| rule.recipe.is_dependency_only())
{
return Err(MISSING_RECIPE_ERROR);
}
if self
.actions
.iter()
.chain(&self.targets)
.any(|target| target.recipe.is_dependency_only() && target.deps.is_blank_content())
{
return Err(MISSING_RECIPE_ERROR);
}
Ok(())
}
}
26 changes: 26 additions & 0 deletions src/ast/string_or_list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,32 @@ impl StringOrList {
Self::List(v) => v.is_empty(),
}
}

/// Report whether the value is the internal dependency-only marker.
///
/// Only `Empty` is a marker; `List(Vec::new())` preserves the explicit
/// empty-command-list validation error.
#[must_use]
pub(crate) const fn is_empty_marker(&self) -> bool {
matches!(self, Self::Empty)
}

/// Report whether the value has no non-whitespace string content.
///
/// This keeps dependency-only validation narrow: executable recipes retain
/// their existing empty-string semantics, while blank dependency templates
/// cannot satisfy the requirement for a real prerequisite.
///
/// For example, a scalar containing only whitespace and a list containing
/// only empty strings are blank; a list containing `"check"` is not.
#[must_use]
pub(crate) fn is_blank_content(&self) -> bool {
match self {
Self::Empty => true,
Self::String(value) => value.trim().is_empty(),
Self::List(values) => values.iter().all(|value| value.trim().is_empty()),
}
}
}

impl From<&str> for StringOrList {
Expand Down
10 changes: 8 additions & 2 deletions src/ir/from_manifest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use std::sync::Arc;

use camino::Utf8PathBuf;

use crate::ast::{NetsukeManifest, Recipe, Rule};
use crate::ast::{MISSING_RECIPE_ERROR, NetsukeManifest, Recipe, Rule};
use crate::localization::{self, keys};

use super::{
Expand All @@ -34,8 +34,14 @@ impl BuildGraph {
/// # Errors
///
/// Returns [`IrGenError`] when a referenced rule is missing, multiple rules
/// are specified for a single target, or no rule is provided.
/// are specified for a single target, no rule is provided, or a directly
/// deserialized manifest violates the recipe contract.
pub fn from_manifest(manifest: &NetsukeManifest) -> Result<Self, IrGenError> {
manifest
.validate_recipes()
.map_err(|_| IrGenError::InvalidManifest {
message: MISSING_RECIPE_ERROR,
})?;
let mut graph = Self::default();
let mut rule_map = IrHashMap::<String, Arc<Rule>>::default();

Expand Down
7 changes: 7 additions & 0 deletions src/ir/graph.rs
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,13 @@ pub struct BuildEdge {
/// ```
#[derive(Debug, Error)]
pub enum IrGenError {
/// Raised when a directly deserialized manifest violates recipe rules.
#[error("{message}")]
InvalidManifest {
/// Stable schema diagnostic identifying the violated recipe rule.
message: &'static str,
},

/// Raised when a target references a rule that is not defined in the
/// manifest.
///
Expand Down
14 changes: 14 additions & 0 deletions src/manifest/load_stage.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
//! Manifest-loading pipeline stage definitions.

/// Stages in the manifest-loading sub-pipeline.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub enum ManifestLoadStage {
/// Read raw manifest content from the filesystem.
ManifestIngestion,
/// Parse raw YAML into a `serde_json::Value` tree.
InitialYamlParsing,
/// Expand `foreach` and `when` template directives.
TemplateExpansion,
/// Deserialize and render string fields into typed manifest data.
FinalRendering,
}
35 changes: 15 additions & 20 deletions src/manifest/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ mod expand;
mod glob;
mod hints;
mod jinja_macros;
mod load_stage;
mod parse_with_config;
mod query;
mod render;
Expand All @@ -52,14 +53,13 @@ mod render;
pub type ManifestValue = serde_json::Value;
/// JSON object mapping string keys to manifest values.
pub type ManifestMap = serde_json::Map<String, ManifestValue>;

pub use diagnostics::{
ManifestError, ManifestName, ManifestSource, map_data_error, map_yaml_error,
};
pub use env_reader::{EnvReadError, EnvReader, process_env_reader};
pub use glob::glob_paths;

pub(crate) use expand::expand_foreach;
pub use glob::glob_paths;
pub use load_stage::ManifestLoadStage;
pub use parse_with_config::from_str_with_env_and_config;
pub(crate) use query::from_path_for_manifest_query;
pub use render::render_manifest;
Expand All @@ -68,19 +68,6 @@ use self::{env_reader::env_var_with, jinja_macros::register_manifest_macros};
#[cfg(test)]
use workspace::open_manifest_workspace;

/// Stages in the manifest-loading sub-pipeline.
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
pub enum ManifestLoadStage {
/// Read raw manifest content from the filesystem.
ManifestIngestion,
/// Parse raw YAML into a `serde_json::Value` tree.
InitialYamlParsing,
/// Expand `foreach` and `when` template directives.
TemplateExpansion,
/// Deserialize and render string fields into typed manifest data.
FinalRendering,
}

/// Invoke the stage callback when present.
fn notify_stage(
on_stage: &mut Option<&mut dyn FnMut(ManifestLoadStage)>,
Expand Down Expand Up @@ -171,11 +158,19 @@ fn from_str_named(
message: localization::message(keys::MANIFEST_PARSE),
})?;

if is_manifest_query {
render::render_manifest_for_manifest_query(manifest, &jinja)
let rendered_manifest = if is_manifest_query {
render::render_manifest_for_manifest_query(manifest, &jinja)?
} else {
render_manifest(manifest, &jinja)
}
render_manifest(manifest, &jinja)?
};
rendered_manifest
.validate_recipes()
.map_err(|detail| ManifestError::Parse {
source: map_data_error(serde_json::Error::custom(detail), name),
message: localization::message(keys::MANIFEST_PARSE),
})?;

Ok(rendered_manifest)
}

/// Translate schema-only recipe errors at the manifest adapter boundary.
Expand Down
3 changes: 2 additions & 1 deletion src/manifest/render.rs
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ fn render_command_recipe(
command: &mut StringOrList,
context: &RecipeRenderContext<'_>,
) -> Result<()> {
if context.mode == RenderMode::ManifestQuery {
if context.mode == RenderMode::ManifestQuery || command.is_empty_marker() {
return Ok(());
}
render_recipe_string_or_list(command, context.env, context.vars, || {
Expand Down Expand Up @@ -313,6 +313,7 @@ thread_local! {
}

#[cfg(test)]
/// Count one recipe-context preparation for the recipe-context tests.
fn record_recipe_context_preparation() {
RECIPE_CONTEXT_PREPARATIONS.with(|count| count.set(count.get() + 1));
}
Expand Down
23 changes: 23 additions & 0 deletions src/manifest/render_command_list_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,29 @@ fn large_command_list_prepares_the_jinja_context_once() {
);
}

#[test]
fn dependency_only_marker_skips_recipe_context_preparation() {
reset_recipe_context_preparations();
let mut command = StringOrList::Empty;
let env = Environment::new();
let vars = Vars::new();
let context = RecipeRenderContext {
env: &env,
vars: &vars,
subject: "target",
mode: RenderMode::Full,
};

render_command_recipe(&mut command, &context)
.expect("dependency-only marker should not need recipe rendering");

assert_eq!(
recipe_context_preparations(),
0,
"dependency-only markers must skip recipe-context preparation"
);
}

#[test]
fn target_recipe_context_reserves_ins_and_outs_placeholders() -> Result<()> {
let env = Environment::new();
Expand Down
Loading
Loading