Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Copy `samplib/httpprm0` to `SYS2.PARMLIB(HTTPPRM0)` and adjust as needed. A mini
```
PORT=8080
DOCROOT=/www
MODULE=MVSMF /zosmf/*
MOD=MVSMF /zosmf/*
```

### Starting and Stopping
Expand Down Expand Up @@ -96,7 +96,7 @@ The server is configured through a Parmlib member referenced by the `HTTPPRM` DD
| `KEEPALIVE_MAX` | 100 | Max requests per connection |
| `LOGIN` | NONE | Authentication mode (NONE / RACF) |
| `SMF` | NONE | SMF recording level |
| `MODULE` | — | Server module registration |
| `MOD` | — | Server module registration |

For the complete reference, see [docs/configuration.md](docs/configuration.md).

Expand All @@ -107,16 +107,19 @@ HTTPD uses a server module system for extending the server with custom functiona
In version 4.0.0, the primary module is [mvsMF](https://github.com/mvslovers/mvsmf), which provides a z/OSMF-compatible REST API for datasets, jobs, and USS files.

```
MODULE=MVSMF /zosmf/*
MOD=MVSMF /zosmf/*
```

Module routing supports both URL prefix matching and file extension matching:
Module routing supports URL prefix matching and automatic extension matching:

```
MODULE=MVSMF /zosmf/* URL prefix — all requests under /zosmf/
MODULE=MYSCRIPT *.lua Extension — files served from DOCROOT
MOD=MVSMF /zosmf/* URL prefix — all requests under /zosmf/
MOD=LUA Extension — *.lua files served from DOCROOT
MOD=REXX Extension — *.rexx files served from DOCROOT
```

Scripting modules like LUA and REXX don't need an explicit pattern — HTTPD derives the file extension from the module name automatically.

Debug modules (HTTPDSRV, HTTPDMTT) are included in the server binary but not enabled by default. They are intended for development and troubleshooting only. See [docs/development.md](docs/development.md) for details on writing your own modules.

## Ecosystem
Expand Down
42 changes: 22 additions & 20 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,40 +68,42 @@ When `LOGIN=RACF`, unauthenticated requests to protected resources receive a red
## Server Modules

```
MODULE=PROGRAM /url/pattern URL prefix match
MODULE=PROGRAM *.ext File extension match
MOD=PROGRAM /url/pattern URL prefix match
MOD=PROGRAM Extension match (derives *.program from name)
```

Server modules are load modules that HTTPD loads at startup via `__load()` and calls directly through the HTTPX function vector. They run inside the server's address space — unlike traditional CGI programs which fork a new process per request. Each module entry maps a URL pattern or file extension to a program name.
Server modules are load modules that HTTPD loads at startup via `__load()` and calls directly through the HTTPX function vector. They run inside the server's address space — unlike traditional CGI programs which fork a new process per request.

| Routing Type | Pattern | Behavior |
|-------------|---------|----------|
| URL prefix | `/zosmf/*` | All requests where the path starts with `/zosmf/` are routed to the module. |
| Extension | `*.lua` | Requests for files with the `.lua` extension are routed to the module. The file path is resolved relative to `DOCROOT`. |
| Routing Type | Syntax | Behavior |
|-------------|--------|----------|
| URL prefix | `MOD=MVSMF /zosmf/*` | All requests where the path starts with `/zosmf/` are routed to the module. |
| Extension | `MOD=LUA` | HTTPD derives the extension `*.lua` from the module name. Requests for `.lua` files are routed to the module. The file path is resolved relative to `DOCROOT`. |

URL prefix matching is checked first. If no prefix matches, extension matching is attempted.

### Available Server Modules

| Module | Pattern | Description |
|--------|---------|-------------|
| MVSMF | `/zosmf/*` | z/OSMF-compatible REST API (datasets, jobs, USS files). Separate project: [mvsmf](https://github.com/mvslovers/mvsmf). |
| HTTPDSRV | `/.dsrv` | Display server status. Debug tool, not for production. |
| HTTPDM | `/.dm` | Display memory. Debug tool, not for production. |
| HTTPDMTT | `/.dmtt` | Display master trace table. Debug tool, not for production. |
| HTTPDSL | `/dsl/*` | Dataset list browser. **Deprecated** — will be replaced by mvsMF. |
| HTTPJES2 | `/jes/*` | JES2 job browser. **Deprecated** — will be replaced by mvsMF. |
| Module | Syntax | Description |
|--------|--------|-------------|
| MVSMF | `MOD=MVSMF /zosmf/*` | z/OSMF-compatible REST API (datasets, jobs, USS files). Separate project: [mvsmf](https://github.com/mvslovers/mvsmf). |
| LUA | `MOD=LUA` | Lua scripting module. Handles `*.lua` files. Separate project (not shipped with 4.0.0). |
| REXX | `MOD=REXX` | REXX scripting module. Handles `*.rexx` files. Separate project (not shipped with 4.0.0). |
| HTTPDSRV | `MOD=HTTPDSRV /.dsrv` | Display server status. Debug tool, not for production. |
| HTTPDM | `MOD=HTTPDM /.dm` | Display memory. Debug tool, not for production. |
| HTTPDMTT | `MOD=HTTPDMTT /.dmtt` | Display master trace table. Debug tool, not for production. |
| HTTPDSL | `MOD=HTTPDSL /dsl/*` | Dataset list browser. **Deprecated** — will be replaced by mvsMF. |
| HTTPJES2 | `MOD=HTTPJES2 /jes/*` | JES2 job browser. **Deprecated** — will be replaced by mvsMF. |

### Example

```
# Production: only mvsMF
MODULE=MVSMF /zosmf/*
MOD=MVSMF /zosmf/*

# Development: mvsMF + debug tools
MODULE=MVSMF /zosmf/*
MODULE=HTTPDSRV /.dsrv
MODULE=HTTPDMTT /.dmtt
MOD=MVSMF /zosmf/*
MOD=HTTPDSRV /.dsrv
MOD=HTTPDMTT /.dmtt
```

## SMF Recording
Expand Down Expand Up @@ -149,6 +151,6 @@ KEEPALIVE_MAX=100
CLIENT_TIMEOUT=10
LOGIN=RACF
TZOFFSET=+01:00
MODULE=MVSMF /zosmf/*
MOD=MVSMF /zosmf/*
SMF=AUTH TYPE=243
```
29 changes: 22 additions & 7 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,17 +138,32 @@ For details on the codepage tables, roundtrip verification, and integration with

### Overview

HTTPD uses a server module system for extensibility. Modules are load modules that run inside the server's address space and are called directly through the HTTPX function vector — unlike traditional CGI programs (as defined by [RFC 3875](https://datatracker.ietf.org/doc/html/rfc3875)) which fork a new process per request and communicate via stdin/stdout.
HTTPD uses a **server module** system for extensibility — not traditional CGI.

> **Note on naming:** Some internal code still uses the legacy name "CGI" (e.g. `httpcgi.h`, `HTTPCGI` struct, `cgimain` entry point). This will be renamed in a future release. The Parmlib keyword has already been changed from `CGI=` to `MODULE=`.
The difference matters: [Traditional CGI](https://publib.boulder.ibm.com/httpserv/manual24/howto/cgi.html) (as defined by [RFC 3875](https://datatracker.ietf.org/doc/html/rfc3875)) forks a new process for each request and communicates via stdin/stdout and environment variables. HTTPD modules are fundamentally different:

| | Traditional CGI | HTTPD Server Modules |
|---|----------------|---------------------|
| Execution | Forked process per request | Loaded into server address space at startup |
| Communication | stdin/stdout, env vars | HTTPX function vector (direct function calls) |
| Request data | `$REQUEST_METHOD`, `$QUERY_STRING` env vars | `http_get_env(httpc, "REQUEST_METHOD")` via HTTPX |
| Response | Write to stdout | `http_resp()`, `http_printf()` via HTTPX |
| Performance | Process fork overhead per request | Direct function call (~10µs) |
| Analogy | Perl/Python CGI scripts | Apache `mod_php`, `mod_rewrite` |

This means you cannot write a standard CGI script (e.g. a REXX program that reads environment variables and writes to stdout) and expect it to work with HTTPD. Server modules must be compiled as MVS load modules and use the HTTPX API.

> **Note on naming:** Some internal code still uses the legacy name "CGI" (e.g. `httpcgi.h`, `HTTPCGI` struct, `cgimain` entry point). This will be renamed in a future release. The Parmlib keyword has already been changed from `CGI=` to `MOD=`.

Modules are registered via the Parmlib:

```
MODULE=MYMODULE /api/* URL prefix routing
MODULE=MYMODULE *.ext Extension-based routing
MOD=MYMODULE /api/* URL prefix routing
MOD=LUA Extension routing (derives *.lua from name)
```

When no pattern is specified, HTTPD derives the file extension from the module name (lowercase). The module then handles all requests for files with that extension in the DOCROOT.

HTTPD loads the module via `__load()` and calls its entry point for matching requests.

### Module Structure
Expand All @@ -171,7 +186,7 @@ int cgimain(HTTPD *httpd, HTTPC *httpc)

### Available Functions (via HTTPX)

server modules call server functions through the HTTPX function vector. Key functions:
Server modules call server functions through the HTTPX function vector. Key functions:

**Response:**
- `http_resp(httpc, code)` — set HTTP status code
Expand Down Expand Up @@ -232,8 +247,8 @@ These modules are built into the HTTPD binary and can be enabled via Parmlib for
Enable them during development:

```
MODULE=HTTPDSRV /.dsrv
MODULE=HTTPDMTT /.dmtt
MOD=HTTPDSRV /.dsrv
MOD=HTTPDMTT /.dmtt
```

Do not enable in production — they expose server internals.
Expand Down
12 changes: 6 additions & 6 deletions docs/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ PORT=8080
MINTASK=3
MAXTASK=9
DOCROOT=/www
MODULE=MVSMF /zosmf/*
MODULE=HTTPLUA *.lua
MOD=MVSMF /zosmf/*
MOD=LUA
```

Copy `samplib/httpprm0` from the distribution to `SYS2.PARMLIB(HTTPPRM0)` as a starting point. The JCL procedure references it via the `HTTPPRM` DD card.
Expand Down Expand Up @@ -98,12 +98,12 @@ See [docs/smf-records.md](smf-records.md) for the record format and field descri

### Extension-Based Module Routing

In addition to URL prefix matching, server modules can now be registered by file extension:
In addition to URL prefix matching (e.g. `MOD=MVSMF /zosmf/*`), scripting modules can be registered without an explicit pattern. HTTPD automatically derives the file extension from the module name:

```
MODULE=HTTPLUA *.lua Any .lua file in DOCROOT → HTTPLUA
MODULE=HTTPREXX *.rexx Any .rexx file in DOCROOT → HTTPREXX
MODULE=MVSMF /zosmf/* All requests under /zosmf/ → MVSMF (unchanged)
MOD=LUA Handles *.lua files from DOCROOT
MOD=REXX Handles *.rexx files from DOCROOT
MOD=MVSMF /zosmf/* All requests under /zosmf/ (explicit prefix)
```

This allows script files to live alongside static content in the normal document root, similar to how Apache handles PHP.
Expand Down
30 changes: 14 additions & 16 deletions samplib/httpprm0
Original file line number Diff line number Diff line change
Expand Up @@ -31,22 +31,20 @@
# DOCROOT=/www
#
# --- Debug ---
# DEBUG=1
#
# --- CGI Modules ---
# Format: CGI=PROGRAM /url/pattern (URL prefix match)
# CGI=PROGRAM *.ext (extension match — uses DOCROOT)
# No CGIs are registered by default.
# Uncomment the lines below to enable them.
#
# CGI=MVSMF /zosmf/*
# CGI=HTTPLUA *.lua
# CGI=HTTPREXX *.rexx
# CGI=HTTPDSRV /.dsrv
# CGI=HTTPDM /.dm
# CGI=HTTPDMTT /.dmtt
# CGI=HTTPDSL /dsl/* (deprecated — use mvsMF dataset API)
# CGI=HTTPJES2 /jes/* (deprecated — use mvsMF jobs API)
# DEBUG=0
#
# --- Modules ---
# Format: MOD=PROGRAM /url/pattern (URL prefix match)
# MOD=PROGRAM *.ext (explicit extension — uses DOCROOT)
# MOD=PROGRAM (extension derived from module name,
# e.g. MOD=LUA → *.lua, MOD=REXX → *.rexx)
#
MOD=MVSMF /zosmf/*
# MOD=HTTPDSRV /.dsrv
# MOD=HTTPDM /.dm
# MOD=HTTPDMTT /.dmtt
# MOD=HTTPDSL /dsl/* (deprecated — use mvsMF dataset API)
# MOD=HTTPJES2 /jes/* (deprecated — use mvsMF jobs API)
#
# --- SMF Recording ---
# SMF=NONE|ERROR|AUTH|ALL [TYPE=nnn]
Expand Down
97 changes: 61 additions & 36 deletions src/httpprm.c
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
static void set_defaults(HTTPD *httpd);
static void parse_line(HTTPD *httpd, char *line);
static void parse_keyvalue(HTTPD *httpd, const char *key, const char *value);
static void parse_mod(HTTPD *httpd, const char *value);
static void parse_login(HTTPD *httpd, const char *value);
static void parse_tzoffset(HTTPD *httpd, const char *value);
static int do_bind(HTTPD *httpd);
Expand Down Expand Up @@ -262,43 +263,12 @@ parse_keyvalue(HTTPD *httpd, const char *key, const char *value)
if (i > 100) i = 100;
httpd->listen_queue = i;
}
else if (strcmp(key, "MOD") == 0) {
parse_mod(httpd, value);
}
else if (strcmp(key, "CGI") == 0) {
/* CGI=PROGRAM /path/* */
char program[9];
char *path;
char *tmp;
int j;
int login = httpd->login & HTTPD_LOGIN_CGI;

tmp = strdup(value);
if (!tmp) return;

/* first token = program name */
path = tmp;
while (*path && *path != ' ' && *path != '\t')
path++;
if (*path) {
*path = '\0';
path++;
while (*path == ' ' || *path == '\t')
path++;
}

/* fold program name to uppercase, max 8 chars */
for (j = 0; j < 8 && tmp[j]; j++)
program[j] = (char)toupper((unsigned char)tmp[j]);
program[j] = '\0';

if (program[0] && path[0]) {
if (!http_add_cgi(httpd, program, path, login)) {
wtof("HTTPD035W Unable to register CGI %s for %s",
program, path);
} else {
wtof("HTTPD036I CGI %s registered for %s", program, path);
}
}

free(tmp);
wtof("HTTPD410W CGI= is deprecated, use MOD= instead");
parse_mod(httpd, value);
}
else if (strcmp(key, "CLIENT_TIMEOUT_MSG") == 0) {
if (atoi(value) > 0) httpd->client |= HTTPD_CLIENT_INMSG;
Expand Down Expand Up @@ -366,6 +336,61 @@ parse_keyvalue(HTTPD *httpd, const char *key, const char *value)
}
}

/* ====================================================================
** Parse MOD=PROGRAM [pattern]
** If pattern is omitted, derive *.<lowercase program> and use DOCROOT.
** ================================================================= */
static void
parse_mod(HTTPD *httpd, const char *value)
{
char program[9];
char auto_pattern[16];
char *path;
char *tmp;
int j;
int login = httpd->login & HTTPD_LOGIN_CGI;

tmp = strdup(value);
if (!tmp) return;

/* first token = program name */
path = tmp;
while (*path && *path != ' ' && *path != '\t')
path++;
if (*path) {
*path = '\0';
path++;
while (*path == ' ' || *path == '\t')
path++;
}

/* fold program name to uppercase, max 8 chars */
for (j = 0; j < 8 && tmp[j]; j++)
program[j] = (char)toupper((unsigned char)tmp[j]);
program[j] = '\0';

/* no pattern → derive *.<lowercase program> */
if (!path[0]) {
auto_pattern[0] = '*';
auto_pattern[1] = '.';
for (j = 0; j < 8 && program[j]; j++)
auto_pattern[2 + j] = (char)tolower((unsigned char)program[j]);
auto_pattern[2 + j] = '\0';
path = auto_pattern;
}

if (program[0]) {
if (!http_add_cgi(httpd, program, path, login)) {
wtof("HTTPD035W Unable to register module %s for %s",
program, path);
} else {
wtof("HTTPD036I Module %s registered for %s", program, path);
}
}

free(tmp);
}

/* ====================================================================
** Parse LOGIN value: NONE, ALL, CGI, GET, HEAD, POST (comma-separated)
** ================================================================= */
Expand Down
Loading