Skip to content

Not able to crawl subpages with crawlSubpages parameter #13

Description

@aleksa-codes

Hello, first of all thanks for this amazing project, I am looking for a way to crawl websites to create assistants for them.

I tried the crawlSubpages parameter on few of my portfolio pages but I don't see subpages, links, on the page being crawled. Am I doing something wrong?

curl 'https://md.dhr.wtf/?url=https://aleksa.codes&crawlSubpages=true&llmFilter=true'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions