# AI Knowledge Base

An AI Knowledge Base turns a website into material an agent can read. You give it a root URL; Stark Infra crawls the page, follows its links, converts everything to Markdown and indexes it for retrieval.

Crawling is asynchronous. The base is created in `processing` status and moves to `success` once the index is ready, or to `failed` when the crawl could not produce one. Poll the base until it leaves `processing`.

Attach the base to an [AI Agent](https://docs.starkinfra.com/api/v2/ai-agent.md) through `knowledgeBaseIds` and the agent consults it before answering: on every message it retrieves the passages most relevant to the question and answers from those.

### The AI Knowledge Base object

**Attributes**

| Name | Type | Description |
| --- | --- | --- |
| `id` | STRING | Unique id for the AI Knowledge Base. |
| `name` | STRING | Name of the knowledge base. Example: "Product Documentation". |
| `rootUrl` | STRING | URL the crawl starts from. Example: "https://docs.starkinfra.com". |
| `isRecursive` | BOOLEAN | Whether the crawl may follow links into other subdomains of the root URL's registered domain. |
| `status` | STRING | Current status of the knowledge base. Options: "processing", "success", "failed". An agent retrieves from a base only once it reaches "success". |
| `tags` | LIST OF STRINGS | Tags associated with the knowledge base. |
| `created` | STRING | Creation datetime. Example: "2022-01-01T00:00:00.000000+00:00". |
| `updated` | STRING | Last update datetime. Example: "2022-01-01T00:00:00.000000+00:00". |

### Create an AI Knowledge Base

`POST /v2/ai-knowledge-base`

Create a base and start crawling it. The call returns immediately with the base in `processing` status.

The crawler seeds itself from the root page, and from the site's `llms.txt` and `sitemap.xml` when they exist. Pages are re-crawled daily, so the index tracks the site as it changes.

**Parameters**

| Name | Required | Description |
| --- | --- | --- |
| `name` | REQUIRED | Name of the knowledge base. Between 1 and 100 characters. |
| `rootUrl` | REQUIRED | URL the crawl starts from. Must be an absolute `http` or `https` URL. |
| `isRecursive` | OPTIONAL | Whether the crawl may leave the root URL's host. Defaults to `true`, which lets it follow links into other subdomains of the same registered domain. Send `false` to index the root host only. |
| `tags` | OPTIONAL | List of up to 100 strings to tag the entity for future queries. |

**Request**

```python
import starkinfra

knowledge_base = starkinfra.aiknowledgebase.create(
    starkinfra.AiKnowledgeBase(
        name="Product Documentation",
        root_url="https://docs.starkinfra.com",
        is_recursive=True,
        tags=["support", "public"]
    )
)

print(knowledge_base)
```

**Response**

```python
AiKnowledgeBase(
    created=2022-01-01 00:00:00,
    id=6767676767676767,
    is_recursive=True,
    name=Product Documentation,
    root_url=https://docs.starkinfra.com,
    status=processing,
    tags=['support', 'public'],
    updated=2022-01-01 00:00:00
)
```

### List AI Knowledge Bases

`GET /v2/ai-knowledge-base`

Get the knowledge bases in your workspace, newest first, in chunks of at most 100, optionally filtered by name, status or id.

**Parameters**

| Name | Required | Description |
| --- | --- | --- |
| `ids` | OPTIONAL | Comma-separated list of ids to get specific entities. When sent, every matching knowledge base is returned at once and no cursor is returned. |
| `name` | OPTIONAL | Case-insensitive substring of the name to filter by. Example: `docs` matches "Product Docs". The filter is applied to each page, so a page may hold fewer than `limit` results, or none, while a `cursor` is still returned. Keep following the cursor until it comes back `null`. |
| `status` | OPTIONAL | Filter by status. Options: `processing`, `success`, `failed`. |
| `limit` | OPTIONAL | Number of results per cursor. Max = 100. |
| `cursor` | OPTIONAL | String used to get the next batch of results, returned by the previous call. |

**Request**

```python
import starkinfra

knowledge_bases = starkinfra.aiknowledgebase.query(
    status="success",
    name="documentation"
)

for knowledge_base in knowledge_bases:
    print(knowledge_base)
```

**Response**

```python
AiKnowledgeBase(
    created=2022-01-01 00:00:00,
    id=6767676767676767,
    is_recursive=True,
    name=Product Documentation,
    root_url=https://docs.starkinfra.com,
    status=success,
    tags=['support', 'public'],
    updated=2022-01-01 00:10:00
)
```

### Get an AI Knowledge Base

`GET /v2/ai-knowledge-base/:id`

Get a single knowledge base by its id. This is the call to poll while the crawl runs.

**Parameters**

| Name | Required | Description |
| --- | --- | --- |
| `id` | REQUIRED | Id of the knowledge base entity. |
| `fields` | OPTIONAL | Comma-separated list of attributes to keep in the response JSON. |

**Request**

```python
import starkinfra

knowledge_base = starkinfra.aiknowledgebase.get("6767676767676767")

print(knowledge_base)
```

**Response**

```python
AiKnowledgeBase(
    created=2022-01-01 00:00:00,
    id=6767676767676767,
    is_recursive=True,
    name=Product Documentation,
    root_url=https://docs.starkinfra.com,
    status=success,
    tags=['support', 'public'],
    updated=2022-01-01 00:10:00
)
```

### Update an AI Knowledge Base

`PATCH /v2/ai-knowledge-base/:id`

Rename a base, retag it or change whether its crawl is recursive. The root URL cannot be changed — point a new base at a new site.

**Parameters**

| Name | Required | Description |
| --- | --- | --- |
| `id` | REQUIRED | Id of the knowledge base entity. |
| `name` | OPTIONAL | New name for the knowledge base. Between 1 and 100 characters. |
| `isRecursive` | OPTIONAL | Whether the next crawl may follow links into other subdomains of the root URL's registered domain. |
| `tags` | OPTIONAL | New list of up to 100 strings to tag the entity. Replaces the current list. |

**Request**

```python
import starkinfra

knowledge_base = starkinfra.aiknowledgebase.update(
    "6767676767676767",
    name="Public Documentation",
    tags=["support"]
)

print(knowledge_base)
```

**Response**

```python
AiKnowledgeBase(
    created=2022-01-01 00:00:00,
    id=6767676767676767,
    is_recursive=True,
    name=Public Documentation,
    root_url=https://docs.starkinfra.com,
    status=success,
    tags=['support'],
    updated=2022-01-02 00:00:00
)
```

### List AI Knowledge Base Hosts

`GET /v2/ai-knowledge-base/:id/hosts`

Get every page the crawler has seen, grouped by host. Each entry carries the page's original URL, the URL of the Markdown file stored for it, and its status: `pending` while the page is still queued, `success` once it has been stored, `failed` when it could not be read.

While a crawl is running this returns the live picture, merged with the last finished one, so you can watch the base fill up page by page.

**Parameters**

| Name | Required | Description |
| --- | --- | --- |
| `id` | REQUIRED | Id of the knowledge base entity. |

**Request**

```python
import starkinfra

hosts = starkinfra.aiknowledgebase.hosts("6767676767676767")

for host, pages in hosts.items():
    print(host)
    for page in pages:
        print(page)
```

**Response**

```python
docs.starkinfra.com
{'original_url': 'https://docs.starkinfra.com/get-started', 'status': 'success', 'storage_url': 'https://storage.googleapis.com/ai-knowledge/6767676767676767/get-started.md'}
{'original_url': 'https://docs.starkinfra.com/api', 'status': 'success', 'storage_url': 'https://storage.googleapis.com/ai-knowledge/6767676767676767/api.md'}
```

### Delete AI Knowledge Bases

`DELETE /v2/ai-knowledge-base`

Delete up to 100 knowledge bases at once. The ids travel in the query string, not in the body.

Agents still referencing a deleted base simply retrieve nothing from it.

**Parameters**

| Name | Required | Description |
| --- | --- | --- |
| `ids` | REQUIRED | Comma-separated list of knowledge base ids to delete. Up to 100 ids. |

**Request**

```python
import starkinfra

knowledge_bases = starkinfra.aiknowledgebase.delete(ids=["6767676767676767"])

for knowledge_base in knowledge_bases:
    print(knowledge_base)
```

**Response**

```python
AiKnowledgeBase(
    created=2022-01-01 00:00:00,
    id=6767676767676767,
    is_recursive=True,
    name=Public Documentation,
    root_url=https://docs.starkinfra.com,
    status=success,
    tags=['support'],
    updated=2022-01-02 00:00:00
)
```

## Available Languages

Code samples on this page are in Python. The same content is available with samples in:

- [Python](https://docs.starkinfra.com/api/v2/ai-knowledge-base-python.md)
- [Node.js](https://docs.starkinfra.com/api/v2/ai-knowledge-base-node.md)
- [PHP](https://docs.starkinfra.com/api/v2/ai-knowledge-base-php.md)
- [Java](https://docs.starkinfra.com/api/v2/ai-knowledge-base-java.md)
- [Ruby](https://docs.starkinfra.com/api/v2/ai-knowledge-base-ruby.md)
- [Elixir](https://docs.starkinfra.com/api/v2/ai-knowledge-base-elixir.md)
- [.NET](https://docs.starkinfra.com/api/v2/ai-knowledge-base-dotnet.md)
- [Go](https://docs.starkinfra.com/api/v2/ai-knowledge-base-go.md)
- [Clojure](https://docs.starkinfra.com/api/v2/ai-knowledge-base-clojure.md)
- [cURL](https://docs.starkinfra.com/api/v2/ai-knowledge-base-curl.md)