-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Add domain skills: youtube/upload, dev-to/publish, google-search-console/check #476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
4b7a3f4
afa309d
322f3e7
6f99dc8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # dev-to/publish — publish an article to DEV.to via the markdown editor (battle-tested 2026-06-28) | ||
|
|
||
| Driving `dev.to/new` with browser-harness against the user's logged-in Chrome. DEV.to gives an **instant, no-queue, high-DR dofollow backlink** in the article body — ideal for cross-posting an existing technical article. Far simpler than rich editors: it's one markdown textarea with Jekyll front matter. | ||
|
|
||
| > Companion skill: `dev-to/scraping.md` (reading dev.to). This file covers writing/publishing. | ||
|
|
||
| ## Pre-flight | ||
| - User logged into dev.to in Chrome. `new_tab("https://dev.to/new")` → if the editor (`textarea#article_body_markdown`) is present you're in; if you see a login wall, pause and ask. | ||
| - The "basic markdown editor" puts **everything in one textarea** — title/tags/description go in YAML front matter at the top, body below. | ||
|
|
||
| ## The flow | ||
|
|
||
| ### 1. Build the content (front matter + markdown body) | ||
| ``` | ||
| --- | ||
| title: "<Title — ALWAYS quote it: a colon or # in the title breaks YAML otherwise>" | ||
| published: true # true = publish now; false = save as draft | ||
| description: "<≤ ~150 chars — quote it too (colons/# are common in descriptions)>" | ||
| tags: programming, ai, webdev, saas # comma-separated, MAX 4, lowercase, no spaces/hyphens inside a tag | ||
| canonical_url: "<optional original URL, if cross-posting, to avoid duplicate-content>" | ||
| --- | ||
|
|
||
| <markdown body> | ||
| ``` | ||
| Quote any user-supplied scalar (title/description/canonical_url). dev.to's front matter is YAML, so an unquoted value containing `:` or a leading `#` will fail the parse or be misread, and the publish silently breaks. | ||
| - **YouTube / Tweet embeds:** liquid tag on its own line — `{% embed https://youtu.be/<id> %}`. | ||
| - Body links are normal markdown `[text](https://...)` and render **dofollow**. | ||
|
|
||
| ### 2. Insert it (React-safe native setter — don't type) | ||
| The editor is one big controlled textarea. Set the value via the native setter + fire input/change (typing char-by-char is slow and flaky): | ||
| ```python | ||
| import json | ||
| md = open("/path/post.md").read() | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
|
||
| js("""(function(){ | ||
| var t=document.querySelector('textarea#article_body_markdown')|| | ||
| [].slice.call(document.querySelectorAll('textarea')).sort((a,b)=>b.getBoundingClientRect().height-a.getBoundingClientRect().height)[0]; | ||
| var setter=Object.getOwnPropertyDescriptor(window.HTMLTextAreaElement.prototype,'value').set; | ||
| setter.call(t, %s); t.dispatchEvent(new Event('input',{bubbles:true})); t.dispatchEvent(new Event('change',{bubbles:true})); | ||
| return 'set:'+t.value.length; | ||
| })()""" % json.dumps(md)) | ||
| ``` | ||
|
|
||
| ### 3. Publish | ||
| - Scroll to the bottom; the button is **"Save changes"** (there is no separate "Publish" button in this editor). With `published: true` in the front matter, clicking **Save changes publishes**; with `published: false` it saves a draft. | ||
| - Success = the URL changes to `https://dev.to/<user>/<slug>`. | ||
|
|
||
| ## Gotchas (field-tested) | ||
| - **"Invalid authenticity token" (CSRF) on save** — the #1 failure. The Rails CSRF token goes stale if the page sat open a while (or across a failed submit). **Fix: `goto("https://dev.to/new")` to reload (fresh token), re-insert the content, save again.** Don't dwell between loading the editor and submitting. | ||
| - **No "Publish" button** — it's "Save changes"; the `published:` front-matter flag decides draft vs live. Don't hunt for a Publish button. | ||
| - **Tags:** max 4, lowercase, each a single token (e.g. `webdev`, not `web-dev`); invalid tags can block the save. | ||
| - **Cover image** is optional but boosts the home-feed; drag-drop only (skip in automation, the user can add later). | ||
| - Cross-posting the same article to multiple sites (HackerNoon, Hashnode, dev.to): set `canonical_url` to the original on the secondary copies if you care about duplicate-content; for pure-backlink plays it's optional. | ||
|
|
||
| ## Why it's worth it | ||
| Instant publish, no moderator queue, DEV.to is high domain authority, and body links are dofollow — so a single cross-post of an existing article = one of the "5 backlinks that matter," in ~3 minutes. Hashnode works the same way (markdown editor + instant publish). | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # google-search-console/check — inspect indexing & submit sitemaps (battle-tested 2026-06-29) | ||
|
|
||
| Driving `search.google.com/search-console` with browser-harness against the user's logged-in Google account. **No API here** unless service-account/OAuth is set up — this is the browser path. GSC is a heavy Angular SPA: use coordinate clicks + `Input.insertText`, and **read reports via screenshots** (text extraction is sparse). | ||
|
|
||
| ## Property URL | ||
| Everything is scoped by `resource_id`. For a **domain property** it's `sc-domain:<domain>` (e.g. `sc-domain:tryskilly.app`). Deep-link directly: | ||
| - Overview: `…/search-console?resource_id=sc-domain:<domain>` | ||
| - Sitemaps: `…/search-console/sitemaps?resource_id=sc-domain:<domain>` | ||
| - Pages (indexing): `…/search-console/index?resource_id=sc-domain:<domain>` | ||
| - Performance: `…/search-console/performance/search_analytics?resource_id=sc-domain:<domain>` | ||
|
|
||
| Confirm login + property by reading the top-left property name and the left nav (Overview / Performance / URL inspection / Pages / Sitemaps…). If it redirects to a Google login, pause and ask the user. | ||
|
|
||
| ## The 4 checks (priority order) | ||
|
|
||
| ### 1. Submit / re-submit the sitemap (highest value after a deploy) | ||
| - Go to the Sitemaps page. The "Add a new sitemap" card has an input + **SUBMIT** button. | ||
| - **GOTCHA — domain properties require the FULL URL.** Entering a relative path (`sitemap-index.xml`) returns **"Invalid sitemap address"**. Enter `https://<domain>/sitemap-index.xml`. | ||
| - **GOTCHA — two inputs on the page.** The top header has an "Inspect any URL" bar AND the card has the sitemap input — both are `<input>`. `document.querySelector('input')` grabs the **wrong** (top) one. Target the sitemap card's input (it sits lower in the main content, ~y 239 at 1080p), then `click` it → `Input.insertText` the full URL → click SUBMIT. | ||
| - Success = a green **"Sitemap submitted successfully"** dialog; the submitted-sitemaps table shows the row with Status **Success**, Last read, and **Discovered pages**. | ||
| - Re-submitting after adding pages nudges a fresh read (Google also re-reads on its own schedule). | ||
|
|
||
| ### 2. Pages (indexing) report — the real "is everything indexed?" answer | ||
| - Two tiles: **Indexed** (count) and **Not indexed (N reasons)**. | ||
| - Scroll to **"Why pages aren't indexed"** for the per-reason table. | ||
| - **GOTCHA — data lags ~2 weeks** ("Last update" date at top). Pages deployed today won't show here for a while; the sitemap submit (step 1) is what accelerates discovery. | ||
|
|
||
| ### 3. URL inspection | ||
| - Top "Inspect any URL" bar → type a full URL → Enter. Shows whether it's on Google + why. Click **"Request indexing"** to push a priority page into the queue (rate-limited, ~10–20/day). | ||
|
|
||
| ### 4. Performance | ||
| - Impressions / clicks / avg position / top queries. Early-stage sites show little — that's expected. | ||
|
|
||
| ## Interpreting "not indexed" reasons | ||
| - **"Discovered – currently not indexed"** → Google found the URL (via sitemap/links) but deprioritized crawling it. This is a **domain-authority / crawl-budget** signal, NOT a technical error. Fix: earn backlinks, strengthen internal linking (hub→spoke), Request-Indexing the key pages, and wait. If most pages sit here, **do not mass-generate more pages** — they'll pile up unindexed; raise authority first. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: The 'Discovered – currently not indexed' guidance is overly definitive and contradicts Google's documentation by ruling out technical causes. Prompt for AI agents |
||
| - **"Crawled – currently not indexed"** → crawled but judged low-value/thin/duplicate. Improve content quality/uniqueness. | ||
| - **"Page with redirect"** → usually benign (trailing-slash or http→https canonicalization); the redirect target is the indexed URL. | ||
| - **"Duplicate without user-selected canonical" / "Alternate page with canonical"** → set/confirm canonical tags. | ||
| - **"Not found (404)"** → a referenced URL is broken; find + fix or remove the reference. | ||
|
|
||
| ## Why no API | ||
| GSC has a Search Console API (sitemaps, URL inspection, search analytics) but it needs a **service account added to the property** or an **OAuth client** — neither is set up here (a prior attempt stalled on service-account propagation). Until that's configured, this browser flow is the way. If you do set it up, the `searchconsole.googleapis.com` URL-inspection + `webmasters` sitemaps endpoints replace steps 1–3. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: The API note overstates automation capabilities: the Search Console URL Inspection API cannot replace the 'Request indexing' click in step 3, and the Indexing API only supports specific schema types (JobPosting/BroadcastEvent). Prompt for AI agents |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # youtube/upload — publish a video to YouTube via Studio (battle-tested 2026-06-28) | ||
|
|
||
| Driving `studio.youtube.com` with browser-harness against the user's logged-in Chrome. This is the durable map of the upload flow — follow it and the first try works. The whole flow is ~5 steps but several controls resist JS selectors and need **coordinate clicks** (noted below). | ||
|
|
||
| ## Pre-flight | ||
| - User must be logged into the target channel in Chrome. Confirm: `page_info()` on `studio.youtube.com` returns `.../channel/<CHANNEL_ID>` and title contains "YouTube Studio". | ||
| - **Check for an existing video first (match by title, not position).** Go to `.../videos/upload` (the full Content list, not just the dashboard's "Latest video" card — that card only shows the most recent one and will miss older duplicates). Scrape each row's title cell **and** its `a[href*="/video/"]` → `/video/(<id>)/` together, then find the row whose title matches the video you intend to publish; build `https://youtu.be/<id>` from that row's id. Never assume the first/most-recent row is the match — confirm the title, or you'll reuse (or skip over) the wrong video. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: Duplicate detection relies on the video title as the only identity key. Because YouTube allows multiple videos with the same title, this can cause the harness to reuse the wrong existing URL or skip a valid upload when titles collide. The instruction should disambiguate using a secondary discriminator (e.g. upload date, duration, source asset) or surface ambiguity for confirmation. Prompt for AI agents
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: Duplicate-check content list doesn't handle pagination — videos on page 2+ are invisible to the scrape, so the agent may still re-upload an existing video when the match is on a later page. Prompt for AI agents |
||
| - Have the absolute path to the local `.mp4` ready. | ||
|
|
||
| ## The flow | ||
|
|
||
| ### 1. Open the upload dialog | ||
| - Click **Create** (top-right): match `aria-label === "Create"` (an `<button>`/`ytcp-button`). Get rect, `click(cx,cy)`. | ||
| - Menu appears with: Create post / **Upload videos** / Go live. Click the element whose trimmed innerText is exactly `Upload videos` (`children.length<=1`, width>0). | ||
|
|
||
| ### 2. Set the file (this triggers the upload immediately) | ||
| - After the dialog opens, find file inputs via CDP and set the **last** one: | ||
| ```python | ||
| doc=cdp("DOM.getDocument", depth=-1, pierce=True) | ||
| nodes=cdp("DOM.querySelectorAll", nodeId=doc["root"]["nodeId"], selector='input[type=file]')["nodeIds"] | ||
| cdp("DOM.setFileInputFiles", files=[ABS_MP4_PATH], nodeId=nodes[-1]) | ||
| ``` | ||
| - Wait ~8s. The dialog shows "Upload complete … Processing will begin shortly". The **public URL is available immediately** in the "Video link" anchor: scrape `a[href*="youtu.be"]` → e.g. `https://youtu.be/qm3aVsWDARY`. Capture it now. | ||
|
|
||
| ### 3. Details — title + description | ||
| - Title and description are **contenteditable `#textbox` divs** (NOT inputs). There are two large ones; sort by `y`: first = Title (pre-filled from filename), second = Description. | ||
| - Replace the title: `click()` it → `document.execCommand('selectAll',false,null)` → `cdp("Input.insertText", text=TITLE)`. | ||
| - Description: `click()` it → `cdp("Input.insertText", text=DESC)`. | ||
| - **Trap:** external links in the description are NOT clickable until the channel completes a one-off verification ("To make external links clickable, first complete a one-off verification"). The link text still shows; fine for a backlink mention. | ||
|
|
||
| ### 4. Audience — "Made for Kids" (REQUIRED, or Next is blocked) | ||
| - Radios are `tp-yt-paper-radio-button`. The "no" option's text is literally **`No, it's not 'Made for Kids'`** (curly quotes around Made for Kids — do NOT match `/not made for kids/`; match `/no,/i` AND `/made for kids/i`). | ||
| - `scrollIntoView({block:'center'})` then `click` its left edge (`rect.x+18, rect.y+height/2`). Verify `aria-checked === "true"`. | ||
|
|
||
| ### 5. Advance + Visibility + Publish | ||
| - Click **Next** (trimmed innerText `Next`, width>0) **3×** with ~3s waits: Details → Video elements → Checks → Visibility. Confirm you're on Visibility (`document.body.innerText` contains "Save or publish"). | ||
| - **Visibility radios resist JS selectors — use COORDINATE clicks.** The three options stack: Private, Unlisted, **Public** (3rd). Selecting any visibility enables the bottom-right button and relabels it `Save`→`Publish`. | ||
| - If a JS query for the radio returns null, click by coordinate. (See coordinate-conversion note below.) | ||
| - Verify selection worked by checking that a `Publish`/`Save` button with `aria-disabled!=="true"` now exists near the bottom (`rect.y>700`). | ||
| - Click **Publish**. Success = a **"Video published"** dialog appears with the share link (`youtu.be/<id>`) + Promote button. | ||
|
|
||
| ## Gotchas (field-tested) | ||
| - **Visibility radios: coordinate clicks only.** `[role=radio]`/`tp-yt-paper-radio-button` text matches fail because the label/description live in sibling nodes. Locate the Public row from a screenshot and click it. | ||
| - **Coordinate conversion:** screenshots come back at 2× retina (e.g. 3840 wide shown at 2000). For `click(x,y)` (CSS px on a ~1920 viewport): `css = displayed_screenshot_coord × 0.96`. | ||
| - **Clicking Publish/Save with no visibility chosen is safely blocked** — a "You need to choose a visibility setting" tooltip shows and the Visibility step turns red; nothing publishes. So a mis-fire here is harmless; just select Public and retry. | ||
| - **Auto-save:** the dialog header reads "Saved as private" / "Saving…" throughout — the video exists as private from the moment of upload; publishing flips it to the chosen visibility. | ||
| - **Don't re-upload an existing video** — check the dashboard card first. | ||
|
|
||
| ## Content best-practices (for the human filling fields) | ||
| - **Title** ≤100 chars, keyword-first. **Description**: 1–2 line hook + links (with timestamps if long). **Tags** in Show More. **Custom thumbnail** beats auto-generated for CTR. Add **end screens / cards** in the "Video elements" step if you skipped it. | ||
| - For embedding in an article (HackerNoon, etc.): paste the `youtu.be/<id>` URL on its own line — most editors auto-embed the player. Public or Unlisted both embed. | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: Double-quote-only YAML guidance is incomplete; embedded quotes and backslashes in user-supplied scalars still break front matter parsing.
Prompt for AI agents
+Quote any user-supplied scalar (title/description/canonical_url). dev.to's front matter is YAML, so an unquoted value containing
:or a leading#will fail the parse or be misread, and the publish silently breaks.{% embed https://youtu.be/<id> %}.[text](https://...)and render dofollow.</file context>