Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ These decisions are final:

These items need further discussion before implementation:

- **Statistics subsystem** — Keep as-is, simplify to minimal counters, or remove? (httpstat.c, httprepo.c, 4-tier time-series arrays ~16 KB)
- ~~**Statistics subsystem**~~**Done.** Replaced with SMF Type 243 records + simple counters (see #54).
- **DSL CGI (httpdsl*.c)** — Clarify scope and naming (Dataset List, not SSI). Keep, refactor, or remove?
- **Demo CGIs (hello.c, abend0c1.c, test.c)** — Remove from build or just disable by default?
- **Lua CGI / REXX CGI** — Code stays, just not registered by default. lua370 remains a dependency.
Expand Down Expand Up @@ -137,7 +137,7 @@ httpget.c → Static file serving with MIME detection
httpresp.c → HTTP response line + headers (currently HTTP/1.0)
httpsend.c → Binary/text data delivery
httpdone.c → Close files
httprepo.c → Log request, update statistics
httprepo.c → Write SMF record, update counters
httprese.c → Reset for next request or close (currently always closes)
httpclos.c → Release HTTPC, close socket
```
Expand Down Expand Up @@ -231,7 +231,7 @@ Missing `DD:HTTPDPRM` → server starts with defaults (port 8080, no CGIs).
- **hello.c, abend0c1.c, test.c**: Demo/test CGIs

**Subsystems:**
- **httpstat.c / httprepo.c**: Statistics (977 LOC) — future TBD
- **httprepo.c**: SMF Type 243 recording + simple counters (~90 LOC)
- **Lua runtime**: lauxlib.c, liolib.c, loadlib.c, httpluax.c (3,297 LOC) — stays for Lua CGI
- **FTP daemon**: ftp*.c (3,290 LOC) — confirmed for removal
- **MQTT telemetry**: HTTPT struct, telemetry_thread — confirmed for removal
Expand Down
22 changes: 11 additions & 11 deletions doc/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ v v v v |
| +------------------+ +--------+ |
| |
| +------------------+ +--------------+ +--------------------+ |
| | httpresp.c | | httpstat.c | | httpcred.c | |
| | Response Headers| | Statistics | | httpauth.c | |
| | httpsend.c | | min/h/d/m | | Credentials/RACF | |
| | httpresp.c | | httprepo.c | | httpcred.c | |
| | Response Headers| | SMF Type243 | | httpauth.c | |
| | httpsend.c | | + Counters | | Credentials/RACF | |
| +------------------+ +--------------+ +--------------------+ |
+------------------------------------------------------------------+
| | |
Expand Down Expand Up @@ -233,7 +233,7 @@ CSTATE_IN -----> CSTATE_PARSE -----> CSTATE_GET/HEAD/PUT/POST/DELETE
| 4. Method | httpget.c etc. | `http_get()` etc. | Open file, determine MIME, serve response |
| 5. Response | httpresp.c | `http_resp()` | Generate HTTP status line + headers |
| 6. Send | httpsend.c | `http_send_*()` | Binary or text data with encoding conversion |
| 7. Report | httprepo.c | `http_report()` | Log request, update statistics |
| 7. Report | httprepo.c | `http_report()` | Write SMF record, update counters |
| 8. Reset | httprese.c | `http_reset()` | Free env vars, clear buffer, prepare for next |

### Dispatch Logic (httppc.c)
Expand Down Expand Up @@ -382,22 +382,22 @@ Telemetry cache (HTTPTC array) stores latest values per topic.
Lua-based configuration loaded from `PARM='CONFIG=dataset(member)'`.
Global Lua tables: `httpd`, `ftpd`, `cgi`, `mqtc`.

### Statistics (httpstat.c)
### Statistics (httprepo.c)

Per-minute/hour/day/month request statistics. Tracks response times
(lowest, highest, average), tally counts, and response codes.
Save/load to MVS dataset in binary format.
SMF Type 243 records written per HTTP request via `smf_write()`.
Simple in-memory counters (total_requests, total_errors,
total_bytes_sent, active_connections) displayed via `/F HTTPD,D S`.

### Console Commands (httpcons.c, httpcmd.c)

Operator interface via `/F HTTPD,command`:
- `D V` — display version
- `D T` — display threads
- `D S [n]` — display statistics
- `D S` — display statistics counters
- `S MAXTASK n` — set max worker threads
- `S MINTASK n` — set min worker threads
- `S LOGIN [all|cgi|none]` — set login requirements
- `S STATS ON|OFF` — control statistics collection
- `S STATS ON|OFF [RESET]` — control SMF recording, reset counters

## Network I/O

Expand Down Expand Up @@ -515,6 +515,6 @@ Build targets:
| Credentials | 20 | httpcred.c, httpauth.c, credentials/src/*.c (18 files) |
| MQTT telemetry | 2 | httppubf.c, httpdmtt.c |
| Debug | 7 | dbgdump.c, dbgenter.c, dbgexit.c, dbgf.c, dbgs.c, dbgtime.c, dbgw.c |
| Statistics | 1 | httpstat.c |
| Statistics/SMF | 1 | httprepo.c |
| Utilities | 16 | hello.c, abend0c1.c, httptest.c, httpclos.c, http1123.c, httpd048.c, httpntoa.c, httpgsna.c, httpgtod.c, httpsecs.c, httprise.c, httpsbz.c, httprbz.c, httpsubt.c, httpcmp.c, httpcmpn.c |
| String/compare | 4 | httpcmp.c, httpcmpn.c, stck2tv.c, httpdbug.c |
9 changes: 5 additions & 4 deletions include/httpcgi.h
Original file line number Diff line number Diff line change
Expand Up @@ -127,11 +127,12 @@ struct httpc {
#define SSI_LEVEL_MAX 10 /* ... max SSI processing level */
UCHAR content_length_set; /* 52 Content-Length was sent */
UCHAR keepalive; /* 53 keep-alive active */
unsigned short request_count; /* 54 requests on this connection */
unsigned short unused3; /* 56 available */
unsigned connect_time; /* 54 SMF time at connect (1/100s) */
unsigned total_bytes_sent; /* 58 accum bytes all requests */
unsigned request_count; /* 5C requests on this connection */

#define CBUFSIZE (0x1000-0x0058) /* ... 4096-88 = 4008 */
UCHAR buf[CBUFSIZE]; /* 58 data buffer */
#define CBUFSIZE (0x1000-0x0060) /* ... 4096-96 = 4000 */
UCHAR buf[CBUFSIZE]; /* 60 data buffer */
}; /* 1000 (4096 bytes) */

/* HTTP mime type */
Expand Down
87 changes: 51 additions & 36 deletions include/httpd.h
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ typedef struct httpm HTTPM; /* HTTP Mime */
typedef struct httpx HTTPX; /* HTTP function vector */
typedef struct httpv HTTPV; /* HTTP Variables */
typedef struct httpcgi HTTPCGI; /* HTTP CGI path and programs */
typedef struct httpstat HTTPSTAT; /* HTTP Statistics record */
typedef struct smf_httpd_request SMF_HTTPD_REQ; /* SMF request */
typedef struct smf_httpd_session SMF_HTTPD_SESS; /* SMF session */
typedef enum cstate CSTATE; /* HTTP Client state */
typedef enum rdw RDW; /* RDW option */

Expand All @@ -78,7 +79,7 @@ struct httpd {
unsigned addr; /* 10 our listener IP address */
int port; /* 14 our listener port */
int listen; /* 18 our listener socket */
FILE *stats; /* 1C statistics file (log) */
void *unused_1c; /* 1C (was: FILE *stats) */

FILE *dbg; /* 20 debug/trace output */
int tzoffset; /* 24 time zone offset in secs */
Expand Down Expand Up @@ -117,20 +118,23 @@ struct httpd {
UCHAR cfg_maxtask; /* 64 config max task */
UCHAR cfg_mintask; /* 65 config min task */
UCHAR cfg_client_timeout; /* 66 client timeout seconds */
UCHAR cfg_st_month_max; /* 67 statistics month records */
UCHAR cfg_st_day_max; /* 68 statistics day records */
UCHAR cfg_st_hour_max; /* 69 statistics hour_records */
UCHAR cfg_st_min_max; /* 6A statistics min records */
UCHAR smf_level; /* 67 SMF recording level */
#define SMF_LEVEL_NONE 0 /* ... no SMF recording */
#define SMF_LEVEL_ERROR 1 /* ... only resp >= 400 */
#define SMF_LEVEL_AUTH 2 /* ... auth events + errors */
#define SMF_LEVEL_ALL 3 /* ... every request + sessions */
UCHAR smf_type; /* 68 SMF record type (def 243) */
UCHAR unused_69[2]; /* 69-6A available */
UCHAR cfg_cgictx; /* 6B # CGI context pointers */
UCHAR ufs_enabled; /* 6C UFS filesystem enabled */
UCHAR dbg_enabled; /* 6D debug output enabled */
UCHAR bind_tries; /* 6E socket bind retry count */
UCHAR bind_sleep; /* 6F socket bind retry delay */
HTTPSTAT **st_month; /* 70 statistics month array */
HTTPSTAT **st_day; /* 74 statistics day array */
HTTPSTAT **st_hour; /* 78 statistics hour array */
HTTPSTAT **st_min; /* 7C statictics min array */
UCHAR *st_dataset; /* 80 statistics load/save dsn */
unsigned total_requests; /* 70 total HTTP requests */
unsigned total_errors; /* 74 total error responses */
unsigned total_bytes_sent; /* 78 total bytes sent */
unsigned active_connections; /* 7C active client connections */
void *unused_80; /* 80 (was: st_dataset) */
UCHAR *cgilua_dataset; /* 84 CGI Lua dataset */
UCHAR *cgilua_path; /* 88 CGI Lua package.path */
UCHAR *cgilua_cpath; /* 8C CGI Lua package.cpath */
Expand Down Expand Up @@ -214,11 +218,12 @@ struct httpc {
#define SSI_LEVEL_MAX 10 /* ... max SSI processing levele*/
UCHAR content_length_set; /* 52 Content-Length was sent */
UCHAR keepalive; /* 53 keep-alive active */
unsigned short request_count; /* 54 requests on this conn */
unsigned short unused3; /* 56 available */
unsigned connect_time; /* 54 SMF time at connect 1/100s*/
unsigned total_bytes_sent; /* 58 accum bytes all requests */
unsigned request_count; /* 5C requests on this conn */

#define CBUFSIZE (0x1000-0x0058) /* ... 4096-88 = 4008 */
UCHAR buf[CBUFSIZE]; /* 50 data buffer */
#define CBUFSIZE (0x1000-0x0060) /* ... 4096-96 = 4000 */
UCHAR buf[CBUFSIZE]; /* 60 data buffer */
/* 1000 */
}; /* 1000 (4096 bytes) */

Expand All @@ -240,21 +245,35 @@ struct httpcgi {
char *pgm; /* 10 external program name */
}; /* 14 (20 bytes) */

struct httpstat {
UCHAR eye[8]; /* 00 eye catcher */
#define HTTPSTAT_EYE "HTTPSTAT" /* ... */
time64_t first; /* 08 first time stamp */
time64_t last; /* 10 last time stamp */
double lowest; /* 18 lowest value */
double highest; /* 20 highest value */
double total; /* 28 accumulated seconds */
unsigned tally; /* 30 number of adds */
short key1; /* 34 key number */
short key2; /* 36 key number */
short resp; /* 38 response code */
short unused; /* 3A unused / available */
unsigned unused2; /* 3C unused / available */
}; /* 40 (64 bytes) */
/* SMF — HTTP records */
#define SMF_TYPE_HTTPD_DEFAULT 243
#define SMF_HTTPD_SUBTYPE_REQ 1 /* Request completed */
#define SMF_HTTPD_SUBTYPE_SESS 2 /* Session closed */

struct smf_httpd_request {
SMF_HEADER hdr; /* 00 Standard SMF Header (18B) */
char subsys[8]; /* 12 Subsystem ID */
short subtype; /* 1A 1 = Request completed */
char userid[8]; /* 1C RACF user (blank=none) */
unsigned client_addr; /* 24 Client IPv4 address */
unsigned resp_code; /* 28 HTTP status code */
unsigned bytes_sent; /* 2C Response bytes */
unsigned duration_us; /* 30 Request duration (us) */
char method[8]; /* 34 GET/POST/PUT/DELETE */
char uri[64]; /* 3C Request URI (truncated) */
}; /* 7C (124 bytes) */

struct smf_httpd_session {
SMF_HEADER hdr; /* 00 Standard SMF Header (18B) */
char subsys[8]; /* 12 Subsystem ID */
short subtype; /* 1A 2 = Session closed */
char userid[8]; /* 1C last RACF user */
unsigned client_addr; /* 24 Client IPv4 address */
unsigned connect_time; /* 28 Connect time (1/100s) */
unsigned duration_us; /* 2C Total session duration us */
unsigned request_count; /* 30 Requests on connection */
unsigned total_bytes; /* 34 Total bytes sent */
}; /* 38 (56 bytes) */

/* HTTP function execution vector */
extern HTTPX *httpx; /* Global pointer to HTTPX */
Expand Down Expand Up @@ -462,12 +481,8 @@ extern int httpcred(HTTPC *httpc) asm("HTTPCRED");
extern int httpd048(HTTPD *httpd) asm("HTTPD048");
extern int http_debug(HTTPC *httpc, const char *options) asm("HTTPDBUG");
extern int http_config(HTTPD *httpd, const char *member) asm("HTTPCONF");
extern int httpstat_add(HTTPD *httpd, HTTPC *httpc) asm("HTTPSTAT");
extern char **httpstat_report(HTTPD *httpd, unsigned months, unsigned days, unsigned hours, unsigned mins) asm("HTTPSTAR");
extern void httpstat_report_free(char ***rpt) asm("HTTPSTAF");
extern void httpstat_clear(HTTPD *httpd) asm("HTTPSTAC");
extern int httpstat_save(HTTPD *httpd, const char *dataset) asm("HTTPSTAS");
extern int httpstat_load(HTTPD *httpd, const char *dataset) asm("HTTPSTAL");
extern void httpsmf(HTTPC *httpc) asm("HTTPSMF");
extern void httpsmf_session(HTTPD *httpd, HTTPC *httpc) asm("HTTPSMFS");
extern HTTPD *cgihttpd(void) asm("CGIHTTPD");
extern HTTPC *cgihttpc(void) asm("CGIHTTPC");
extern int http_getc(HTTPC *httpc) asm("HTTPGETC");
Expand Down
16 changes: 9 additions & 7 deletions samplib/httpprm0
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,12 @@
# CGI=HTTPLUA /lua/*
# CGI=HTTPREXX /rexx/*
#
# --- Statistics ---
# CLIENT_STATS=1
# CLIENT_STATS_MONTH_MAX=24
# CLIENT_STATS_DAY_MAX=60
# CLIENT_STATS_HOUR_MAX=48
# CLIENT_STATS_MINUTE_MAX=120
# CLIENT_STATS_DATASET=
# --- SMF Recording ---
# SMF=NONE|ERROR|AUTH|ALL [TYPE=nnn]
# NONE - no SMF recording (default)
# ERROR - write SMF on HTTP errors (resp >= 400)
# AUTH - write on auth events (401/403) and errors
# ALL - every request (subtype 1) + session close (subtype 2)
# TYPE - SMF record type (128-255, default 243)
# SMF=NONE
# SMF=ALL TYPE=200
7 changes: 7 additions & 0 deletions src/httpclos.c
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,13 @@ httpclos(HTTPC *httpc)
unlock(httpd,0);

if (httpc) {
if (httpd->active_connections > 0)
httpd->active_connections--;

// Write SMF session record (subtype 2) on final disconnect
if (httpd->smf_level == SMF_LEVEL_ALL)
httpsmf_session(httpd, httpc);

/* make sure we closed the file */
if (httpc->fp) http_done(httpc);
if (httpc->ufp) ufs_fclose(&httpc->ufp);
Expand Down
Loading
Loading