Connect PipesHub to Impossible Cloud Storage and Inference

Interactive demo

The demo below walks through the full setup, from creating the bucket to asking the first questions. The steps that follow describe the same configuration in text form.

How to Build an AI-Searchable Data Lake on European Infrastructure with Impossible Cloud S3 Storage, Impossible Cloud Managed Inference, and PipesHub. The demo follows a new customer, Suspiciously Good Sensors Ltd., as they store business documents and run AI on them entirely on European infrastructure. It covers setting up Impossible Cloud S3 Storage as a data lake, connecting Impossible Cloud Managed Inference to an AI app, and getting answers with sources from company documents stored in Europe. Create the data lake: Create a bucket in Impossible Cloud (IC) S3 storage, which serves as the company data lake. Click the Bucket Name field, name the bucket, and select a European storage region. The full list of regions is in the Impossible Cloud documentation. In this example the region is eu-north-1 (Denmark). Click Add to create the bucket, named sgs-company-data. Open the empty bucket and upload the business documents via the S3 API. The upload includes invoices, contracts, policies and reports. Connect PipesHub to Impossible Cloud Managed Inference: PipesHub is an open-source AI app. Open the workspace settings and click AI Models, where all AI models are configured in one place. Impossible Cloud's managed inference endpoint is OpenAI-compatible, so click Configure on OpenAI Compatible. Three models are added. For each, enter a friendly name, the endpoint URL from the Impossible Cloud inference documentation, an API key, and the exact model ID from the Impossible Cloud catalogue, then click Add Model. Chat model: qwen3.8-27b, a reasoning model that thinks through questions and can also read images. Reasoning and Multimodal both stay switched on. Indexing model: qwen3-30b-instruct, a smaller, faster and cheaper model that summarises and classifies every document. Reasoning and Multimodal are switched off because the model answers directly and only reads text, which keeps indexing fast. Models appear under Configured Models. Select IC - qwen3-30b-instruct as the indexing model. Embedding model: Open the For Embedding tab and add qwen3-embedding-8b through OpenAI Compatible using the same endpoint. The embedding model turns documents into searchable vectors. It appears under Configured Models. Connect the bucket as a data source: Open Connectors and use PipesHub's S3-compatible connector, listed as MinIO. Name the connection Impossible Cloud Data Lake. Paste the S3 endpoint of your region, shown in the bucket view of the Impossible Cloud console; the full list is in the storage documentation. Enter the access key and secret key, which are created in the Impossible Cloud console under IAM -> Keys. Click Next to reach the sync options. Sync can run on a schedule, so new documents are picked up automatically, or manually. This demo uses Manual. Optional sync filters can be added; empty filters sync the whole data lake. Click Save Configuration, then Start Syncing Now. PipesHub reads all files from the bucket and indexes them with the Impossible Cloud hosted models. The indexing status page shows all 40 documents indexed and searchable, with none failed. Ask questions on company data: Click Back to app to start chatting. The plus button selects the sources for each question: internal data lake, web search, or both. With Internal Search and Web Search both switched off, the model cannot answer who Suspiciously Good Sensors' top customer was, because customer knowledge lives in the data lake. With the data lake enabled, the same question returns a full ranking of all customers with the source document linked. A follow-up asks whether Harbor Logistics was invoiced according to contract terms. The AI compares two documents and finds a billing error: Harbor was overcharged by EUR 2,614.98, and it recommends a credit note. This shows AI running on a company's own data lake, with storage and inference from Impossible Cloud, a European cloud provider. Documents are stored in Impossible Cloud S3 Storage and Managed Inference is added on top, so the data lake and the AI come from one European provider.

Prerequisites

A running PipesHub instance with admin access. To install PipesHub, see the PipesHub Quickstart.
An Impossible Cloud Storage account
An Impossible Cloud inference account and API key
The documents you want to make searchable, for example PDFs

Step 1: Create a bucket and upload your documents

In the Impossible Cloud Storage Console, go to Buckets and click Add Bucket.
Enter a Bucket Name, for example sgs-company-data, and select a Region, for example eu-north-1.
Click Add.
Open the bucket and upload your documents with Upload File, or use any S3-compatible tool or script.

Step 2: Get your inference endpoint URL, API key and model IDs

You can find your endpoint URL in the Impossible Cloud console, and create an API key there. The model IDs are listed in the Impossible Cloud model catalogue.

PipesHub uses three models with different jobs. This article uses the following examples:

Chat model: qwen3.8-27b, a reasoning model that answers questions
Indexing model: qwen3-30b-instruct, a fast model that summarizes and classifies documents during indexing
Embedding model: qwen3-embedding-8b, which turns text into vectors for semantic search

Step 3: Add the chat model in Pipeshub

In PipesHub, open Workspace Settings and go to AI Models.
On the Model Providers tab, select For LLMs, find OpenAI Compatible and click Configure.
Model Friendly Name: enter a name your team will recognize, for example IC - qwen3.8-27b.
Endpoint URL: enter your Impossible Cloud inference endpoint URL, including /v1.
API Key: paste your Impossible Cloud API key.
Model Name: enter the exact model ID, for example qwen3.8-27b.
Under Model Compatibilities, turn Reasoning on. Turn Multimodal on only if the model accepts image input and you want PipesHub to send images.
Click Add Model.

The first LLM you add becomes the default model for chat. For details on all fields, see the PipesHub documentation for OpenAI Compatible LLMs.

Step 4: Add and assign the indexing model

On the For LLMs tab, click Configure on OpenAI Compatible again.
Fill in the form as in Step 3, with a different model: for example IC - qwen3-30b-instruct as the friendly name and qwen3-30b-instruct as the Model Name.
Turn Reasoning and Multimodal off.
Click Add Model.
Under Model Role Assignments, open the Indexing Model dropdown and select the indexing model you just added.

If you skip the assignment, PipesHub uses the chat model for indexing, which is slower and more expensive for large document sets.

Step 5: Add the embedding model

Switch to the For Embedding tab, find OpenAI Compatible and click Configure.
Model Friendly Name: for example IC - qwen3-embedding-8b.
Endpoint URL and API Key: the same values as in Step 3.
Model Name: qwen3-embedding-8b.
Leave Output Dimensions empty to use the model default, and keep Multimodal off.
Click Add Model.

Note: Add the embedding model before you connect the bucket. Once documents are indexed, you cannot switch to a different embedding model without removing the indexed documents. For details, see the PipesHub documentation for OpenAI Compatible embedding models.

Step 6: Connect the bucket with the MinIO connector

PipesHub connects to S3-compatible storage such as Impossible Cloud through its MinIO connector.

In the Impossible Cloud Storage Console, go to IAM > Keys and create an access key for PipesHub. PipesHub only reads from the bucket, so read access to the bucket is sufficient.
In PipesHub, go to Workspace Settings > Connectors, search for MinIO and click Setup.
On the Authenticate Instance tab, enter:
- Instance name: for example Impossible Cloud Data Lake
- Endpoint URL: the S3 endpoint of your bucket's region from Step 1, for example https://eu-north-1.storage.impossibleapi.net
- Access Key and Secret Key: the access key you just created
Keep Use SSL/TLS and Verify SSL Certificate enabled.
Click Next.

For details on the connector, see the PipesHub connectors documentation.

Step 7: Choose the sync strategy and run the first sync

On the Configure Records tab, choose a Sync Strategy:
- Scheduled: PipesHub syncs the bucket at the Sync Interval you select, so new documents are picked up automatically.
- Manual: PipesHub syncs only when you trigger it.
Optional: under Sync filters, limit the sync to certain File Extensions or Folders. Leave both empty to sync the whole bucket.
Click Save Configuration, then Start Syncing Now.
Open the connector instance to follow the Records Status. Once all records show as Completed, the documents are indexed and searchable.

Step 8: Ask questions about your data

Click Back to app.
Click + below the chat input and make sure Internal Search is enabled. Turn Web Search off to get answers only from your own documents.
Ask a question in plain language. PipesHub retrieves the relevant passages from your documents and the chat model answers with the source documents listed below the answer.