Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
e8b6f1e
fix: update Bitbucket clone auth to use API token scheme (#509)
octo-patch Apr 21, 2026
5b43df5
Deepwiki is coming back
sng-asyncfunc Apr 21, 2026
04e7c13
Add Deepwiki-Open 2.0 section to README
sng-asyncfunc May 16, 2026
bba5b6c
New Feature: Add optional LiteLLM Docker Compose setup for local LLM …
marazik May 22, 2026
56669d8
Grok Wiki is now Live!
sng-asyncfunc May 22, 2026
892b0cc
exclude golang vendor dir (#531)
halfcrazy Jun 2, 2026
21e7033
Preload tiktoken encoding cache in Docker images (#528)
halfcrazy Jun 2, 2026
16f35a0
Feature: Introduce LiteLLM client for multi-provider model routing (#…
marazik Jun 3, 2026
7417662
chore(sync): merge upstream deepwiki updates
RichelynScott Jul 2, 2026
ae2471d
feat(provider): add direct minimax m3 lane
RichelynScott Jul 2, 2026
da17699
docs(provider): record minimax live smoke
RichelynScott Jul 2, 2026
5c07e9a
Merge remote-tracking branch 'origin/main' into sync/upstream-20260701
RichelynScott Jul 2, 2026
7fa3f90
Merge remote-tracking branch 'origin/main' into sync/upstream-20260701
RichelynScott Jul 2, 2026
06285e4
Merge remote-tracking branch 'origin/main' into sync/upstream-20260701
RichelynScott Jul 2, 2026
5375cbb
fix(api): register websocket route compatibly
RichelynScott Jul 2, 2026
4149a7f
Merge remote-tracking branch 'origin/main' into sync/upstream-20260701
RichelynScott Jul 2, 2026
25ee9ce
fix(provider): handle minimax cumulative stream chunks
RichelynScott Jul 2, 2026
0f9e2cc
fix(provider): preserve minimax incremental stream chunks
RichelynScott Jul 2, 2026
b12ed8a
Merge remote-tracking branch 'origin/main' into sync/upstream-20260701
RichelynScott Jul 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.sample
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
GOOGLE_API_KEY=your-google-api-key-here
# OPENAI_API_KEY=your-openai-api-key-here
# OPENROUTER_API_KEY=your-openrouter-api-key-here
# MINIMAX_API_KEY=your-minimax-api-key-or-subscription-key-here
# MINIMAX_BASE_URL=https://api.minimax.io/v1

# === Optional: Azure OpenAI ===
# AZURE_OPENAI_API_KEY=
Expand Down
118 changes: 118 additions & 0 deletions .research/deep-research-deepwiki-open-sync-evaluation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Deep Research Report: DeepWiki Open Local Fork, Upstream Sync, and OpenDeepWiki Comparison

**Generated**: 2026-07-01
**Research Parameters**: Breadth 3, Depth 1
**Sources Consulted**: local git/source, GitHub `gh` metadata/API, hosted DeepWiki MCP direct streamable HTTP probes, Codex-native explorer agents

## Executive Summary

The local `RichelynScott/deepwiki-open` fork is a focused self-hosted
DeepWiki-Open integration with a custom FastMCP stdio wrapper around a local
Dockerized FastAPI/Next.js app. Its highest-value customizations are local path
translation, stream aggregation, usage/cost telemetry, stale-cache warnings, and
ephemeral-file exclusion for RAG citation hygiene.

AsyncFuncAI upstream has 8 commits after the local merge-base `05591ee`. The
sync is technically clean and low-conflict, adding LiteLLM support, Docker
tiktoken cache preloading, vendor-dir exclusion, and docs updates. It is not
security-clean for broad deployment: open public reports cover unauthenticated
file-read/git-clone SSRF, dependency advisories, and missing private disclosure
workflow. The sync branch is therefore suitable for review/local use, not for
network exposure without hardening.

`AIDotNet/OpenDeepWiki` is a more complete self-hosted knowledge-base platform,
not a drop-in replacement. It has releases, database-backed admin/user/API-key
features, background workers, and built-in MCP endpoints, but also a larger
.NET/Next.js/database/runtime surface that should be evaluated in isolation.

## Detailed Findings

### Local Fork

- Local checkout started clean on `main` and aligned with `origin/main`.
- The repo was initially shallow; `git fetch --unshallow origin` was required
before `git merge-base HEAD upstream/main` returned `05591ee`.
- Fork-owned seams are `mcp_wrapper.py`, `docker-compose.override.yml`,
`requirements-mcp.txt`, `.env.sample`, `CLAUDE.md`, `FYI.md`, and `docs/`.
- The no-delete static gate found `Path.unlink()` in `mcp_wrapper.py`; the sync
branch changes force-refresh cache invalidation to move old `.pkl` files into
`~/.adalflow/databases/archive/` instead.

### AsyncFuncAI Upstream

- `gh repo view AsyncFuncAI/deepwiki-open` reported MIT license, default branch
`main`, 17,113 stars, 1,922 forks, no latest release, pushed 2026-06-03.
- Hosted DeepWiki MCP became available after live Codex config repair. Direct
streamable HTTP probes confirmed server `DeepWiki` version `2.14.3`, listed
`read_wiki_structure`, `read_wiki_contents`, and `ask_question`, and returned
an AsyncFuncAI wiki structure covering overview, backend architecture, RAG
pipeline, frontend architecture, API reference, tests, and glossary.
- Hosted DeepWiki MCP corroborated that AsyncFuncAI is a FastAPI + Next.js RAG
app using local FAISS-style storage, repository cloning, configurable
Google/OpenAI/OpenRouter/AWS Bedrock/Ollama/Azure providers, permissive CORS,
and exposed backend/frontend ports.
- The merged upstream commits add LiteLLM files (`api/litellm_client.py`,
`Dockerfile-litellm`, `docker-compose-litellm.yml`, `litellm-config.yml`) and
tiktoken cache preload in `Dockerfile`.
- Open upstream issues include security and dependency reports, notably
unauthenticated file-read/git-clone SSRF and dependency advisories.
- `npm audit --audit-level=moderate --json` in the sync worktree reported 23
vulnerabilities: 17 moderate, 5 high, and 1 critical.

### AIDotNet/OpenDeepWiki

- `gh repo view AIDotNet/OpenDeepWiki` reported MIT license, default branch
`main`, latest release `v2.0.3` published 2026-05-30, pushed 2026-06-13,
3,376 stars, and 432 forks.
- Hosted DeepWiki MCP returned an AIDotNet wiki structure covering architecture,
AI document generation, repository management, chat assistant, MCP, admin UI,
backend API/services, infrastructure, and development guide.
- Hosted DeepWiki MCP corroborated that AIDotNet is a .NET 10 + Next.js 16
platform with Docker Compose install, AI provider/model binding, Git/ZIP/local
ingestion, MCP endpoints, permissive CORS, local path root allowlisting, and
plaintext `AuthPassword` repository credential storage.
- README/source inspection shows .NET 10 backend, Next.js 16/React 19 frontend,
SQLite/PostgreSQL, LibGit2Sharp repo processing, background workers, admin
console, users/roles/departments/API keys, chat, and MCP endpoints.
- It has better product completeness and release discipline than AsyncFuncAI,
but introduces heavier runtime and security review requirements.
- High-risk defaults or surfaces include default JWT/admin bootstrap,
permissive CORS, root compose runtime, plaintext private repo credential
fields, disabled Git TLS certificate checks, ZIP/local ingestion, and Graphify
subprocess execution.

## Key Data Points

| Item | Value |
|---|---|
| Local merge-base | `05591ee7160fcc8eeb4d976d10ce566a16c2eb94` |
| Local ahead / upstream behind | local 17 ahead, upstream 8 ahead before sync |
| Sync branch | `sync/upstream-20260701` |
| Sync commit | current branch tip; run `git rev-parse HEAD` |
| Sync worktree | `/home/riche/MCPs/deepwiki-open/WORKTREES/upstream-20260701` |
| AsyncFuncAI stars/forks | 17,113 / 1,922 |
| AIDotNet stars/forks | 3,376 / 432 |
| npm audit count | 23 total: 17 moderate, 5 high, 1 critical |

## Emerging Trends And Contrarian Views

- The local wrapper is becoming more valuable than the upstream app for agent
workflows because it adds operational safety and cost telemetry that upstream
does not provide.
- AIDotNet is attractive if the goal changes from single-user local RAG to
managed team knowledge-base service. For the current local MCP use case, its
complexity is a liability until isolated evaluation proves otherwise.
- Upstream parity is less important than hardening the ingestion boundary:
bind scope, auth, CORS, local path policy, SSRF defenses, dependency updates,
and token handling should drive the next branch.

## Sources

- Local repository: `/home/riche/MCPs/deepwiki-open`
- Sync worktree: `/home/riche/MCPs/deepwiki-open/WORKTREES/upstream-20260701`
- Hosted DeepWiki MCP: `https://mcp.deepwiki.com/mcp`
- AsyncFuncAI repo: https://github.com/AsyncFuncAI/deepwiki-open
- AsyncFuncAI issue list: https://github.com/AsyncFuncAI/deepwiki-open/issues
- AIDotNet repo: https://github.com/AIDotNet/OpenDeepWiki
- AIDotNet releases: https://github.com/AIDotNet/OpenDeepWiki/releases
- Review report: `docs/DEEPWIKI_OPEN_SYNC_EVALUATION_20260701.md`
3 changes: 3 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,12 @@ RUN if [ -n "${CUSTOM_CERT_DIR}" ]; then \
fi

ENV PATH="/opt/venv/bin:$PATH"
ENV TIKTOKEN_CACHE_DIR=/opt/tiktoken_cache/

# Copy Python dependencies
COPY --from=py_deps /api/.venv /opt/venv
RUN mkdir -p "$TIKTOKEN_CACHE_DIR" && \
python -c "import tiktoken; tiktoken.get_encoding('cl100k_base')"
COPY api/ ./api/

# Copy Node app
Expand Down
125 changes: 125 additions & 0 deletions Dockerfile-litellm
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# syntax=docker/dockerfile:1-labs

FROM node:20-alpine AS node_base

FROM node_base AS node_deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --legacy-peer-deps

FROM node_base AS node_builder
WORKDIR /app
COPY --from=node_deps /app/node_modules ./node_modules
COPY --exclude=./api . .
RUN NODE_ENV=production npm run build

FROM python:3.11-slim AS py_deps
WORKDIR /api
COPY api/pyproject.toml .
COPY api/poetry.lock .
RUN python -m pip install poetry==2.0.1 --no-cache-dir && \
poetry config virtualenvs.create true --local && \
poetry config virtualenvs.in-project true --local && \
poetry config virtualenvs.options.always-copy --local true && \
POETRY_MAX_WORKERS=10 poetry install --no-interaction --no-ansi --only main && \
poetry cache clear --all .

FROM python:3.11-slim AS ollama_base
RUN apt-get update && apt-get install -y --no-install-recommends \
curl zstd && rm -rf /var/lib/apt/lists/*

# Detect architecture and download appropriate Ollama version
# ARG TARGETARCH can be set at build time with --build-arg TARGETARCH=arm64 or TARGETARCH=amd64
ARG TARGETARCH
RUN OLLAMA_ARCH="" && \
if [ "$TARGETARCH" = "arm64" ]; then \
echo "Building for ARM64 architecture." && \
OLLAMA_ARCH="arm64"; \
elif [ "$TARGETARCH" = "amd64" ]; then \
echo "Building for AMD64 architecture." && \
OLLAMA_ARCH="amd64"; \
else \
echo "Error: Unsupported architecture '$TARGETARCH'. Supported architectures are 'arm64' and 'amd64'." >&2 && \
exit 1; \
fi && \
(set -o pipefail; \
curl -fL "https://ollama.com/download/ollama-linux-${OLLAMA_ARCH}.tar.zst" \
| zstd -d | tar -x -C /usr)

RUN ollama serve > /dev/null 2>&1 & \
sleep 20 && \
ollama pull nomic-embed-text && \
ollama pull qwen3:1.7b

# Use Python 3.11 as final image
FROM python:3.11-slim

# Set working directory
WORKDIR /app

# Install Node.js and npm
RUN apt-get update && apt-get install -y \
curl \
gnupg \
git \
ca-certificates \
&& mkdir -p /etc/apt/keyrings \
&& curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \
&& echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list \
&& apt-get update \
&& apt-get install -y nodejs \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

ENV PATH="/opt/venv/bin:$PATH"
ENV TIKTOKEN_CACHE_DIR=/opt/tiktoken_cache/

# Copy Python dependencies
COPY --from=py_deps /api/.venv /opt/venv
RUN mkdir -p "$TIKTOKEN_CACHE_DIR" && \
python -c "import tiktoken; tiktoken.get_encoding('cl100k_base')"
COPY api/ ./api/

# Copy Node app
COPY --from=node_builder /app/public ./public
COPY --from=node_builder /app/.next/standalone ./
COPY --from=node_builder /app/.next/static ./.next/static
COPY --from=ollama_base /usr/bin/ollama /usr/local/bin/
COPY --from=ollama_base /root/.ollama /root/.ollama

# Expose the port the app runs on
EXPOSE ${PORT:-8001} 3000

# Create a script to run both backend and frontend
RUN echo '#!/bin/bash\n\
# Start ollama serve in background\n\
ollama serve > /dev/null 2>&1 &\n\
\n\
# Load environment variables from .env file if it exists\n\
if [ -f .env ]; then\n\
export $(grep -v "^#" .env | xargs -r)\n\
fi\n\
\n\
# Check for required environment variables\n\
if [ -z "$OPENAI_API_KEY" ] || [ -z "$GOOGLE_API_KEY" ]; then\n\
echo "Warning: OPENAI_API_KEY and/or GOOGLE_API_KEY environment variables are not set."\n\
echo "These are required for DeepWiki to function properly."\n\
echo "You can provide them via a mounted .env file or as environment variables when running the container."\n\
fi\n\
\n\
# Start the API server in the background with the configured port\n\
python -m api.main --port ${PORT:-8001} &\n\
PORT=3000 HOSTNAME=0.0.0.0 node server.js &\n\
wait -n\n\
exit $?' > /app/start.sh && chmod +x /app/start.sh

# Set environment variables
ENV PORT=8001
ENV NODE_ENV=production
ENV SERVER_BASE_URL=http://localhost:${PORT:-8001}

# Create empty .env file (will be overridden if one exists at runtime)
RUN touch .env

# Command to run the application
CMD ["/app/start.sh"]
3 changes: 3 additions & 0 deletions Dockerfile-ollama-local
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,12 @@ RUN apt-get update && apt-get install -y \
&& rm -rf /var/lib/apt/lists/*

ENV PATH="/opt/venv/bin:$PATH"
ENV TIKTOKEN_CACHE_DIR=/opt/tiktoken_cache/

# Copy Python dependencies
COPY --from=py_deps /api/.venv /opt/venv
RUN mkdir -p "$TIKTOKEN_CACHE_DIR" && \
python -c "import tiktoken; tiktoken.get_encoding('cl100k_base')"
COPY api/ ./api/

# Copy Node app
Expand Down
Loading