Skip to content

doctk - Document Toolkit

A composable, functional toolkit for structured document manipulation with interactive REPL, VS Code extension, and Language Server support.

Tests Python 3.12+ License: MIT Code style: ruff

Version: 0.1.0-dev (MVP Complete - Pre-Release)

Philosophy

Inspired by category theory, set theory, and the Zen of Python, doctk provides elegant primitives for document transformation:

  • Composable: Complex operations built from simple primitives
  • Pure: Operations transform rather than mutate
  • Format-agnostic: Operates on universal document AST
  • Type-safe: Well-typed transformations
  • Pipeable: Unix-philosophy pipelines for documents

Features

Core Engine

  • 📚 Core abstractions - Document, Node hierarchy, operations
  • 🔧 Markdown support - Full parser and writer with markdown-it-py
  • 🌲 Document outliner - Tree visualization with rich console output
  • ✅ 749 tests passing - Comprehensive test coverage (68.78%)
  • 🔒 Type-safe - Full type annotations throughout

Operations

  • 🔍 Selection & filtering - select, where predicates
  • ↕️ Level operations - promote, demote, lift, lower
  • 🔄 Structure operations - nest, unnest, move_up, move_down
  • ⚡ Pipe syntax - doc | select(heading) | promote()
  • 🧩 Composable - Chain operations functionally

DSL and Execution

  • 💬 Domain-Specific Language - Intuitive document manipulation DSL
  • 🖥️ Interactive REPL - Live document editing with state management
  • 📄 Script execution - Run .tk files with doctk commands
  • 📝 Code blocks - Execute DSL from Markdown documents
  • ⚠️ Error recovery - Line/column position tracking

VS Code Extension

  • 🌲 Tree view - Hierarchical document outline sidebar
  • 🖱️ Drag-and-drop - Visual section reorganization
  • ⌨️ Keyboard shortcuts - Fast operations (promote, demote, move, delete, rename)
  • 🎨 Context menus - Right-click operations
  • 🔄 Real-time sync - Bidirectional editor synchronization
  • ⚡ Performance - Optimized for 1000+ heading documents
  • 📦 Ready to install - Packaged as .vsix

Language Server (LSP)

  • ✨ Auto-completion - Operation suggestions as you type
  • 📖 Hover docs - Rich documentation with examples
  • 🔍 Syntax validation - Real-time diagnostics
  • 💡 Signature help - Parameter hints and types
  • 🗺️ Document symbols - Navigate operations easily
  • 🤖 AI support - Structured operation catalog for AI agents
  • ⚡ Fast - < 200ms response times

Integration & Architecture

  • 🔌 JSON-RPC bridge - TypeScript ↔ Python communication
  • 🏗️ Pluggable - Extensible architecture for new interfaces
  • ✏️ Granular edits - Cursor position preservation
  • 🆔 Centralized IDs - Consistent node identification
  • 💾 Memory management - LRU cache for large documents
  • 🛡️ Error resilience - Automatic recovery and retry
  • 🧪 E2E tested - Comprehensive integration tests

Quick Examples

Python API

from doctk import Document
from doctk.operations import select, where, promote, demote
from doctk.integration import StructureOperations

# Load document
doc = Document.from_file("guide.md")

# Pipe-style transformations
result = doc | select(heading) | where(level=3) | promote()

# Save
result.to_file("guide_updated.md")

# Structure operations (static methods)
result = StructureOperations.move_up(doc, node_id="h2-intro")
result = StructureOperations.nest(doc, node_id="h3-details", under_id="h2-intro")

Interactive REPL

# Start REPL
$ doctk repl guide.md

# Execute operations
doctk> promote(1)        # Promote first heading
doctk> move_up(2)        # Move second section up
doctk> nest(3, under=1)  # Nest section 3 under section 1
doctk> save output.md    # Save changes

Script Execution

# script.tk - Reorganize document structure
promote(1)
move_up(2)
nest(3, under=1)

Execute with: doctk execute script.tk guide.md

Note: The script modifies the input file in place.

Documentation Structure

Getting Started

Development

API Reference

Guides

Design & Architecture

Design documents are available in the repository but not included in the published docs site:

  • docs/design/02-pluggable-architecture.md - Current architecture overview
  • docs/design/03-core-integration-decisions.md - Architecture Decision Records (ADRs)
  • docs/design/04-lsp-architecture.md - Language Server design
  • docs/design/01-initial-design.md - ⚠️ Historical: Original vision (superseded by specs)

Specifications

For current feature specifications and implementation tracking, see:

  • .kiro/specs/ - Structured Kiro specs (requirements, design, tasks)
  • .kiro/steering/ - Project conventions and AI assistant guidance

Historical Archive

Historical documents are available in the repository at docs/archive/:

  • POC summary, session notes, original specification, reproduction guide

Roadmap

🚀 Planned for v0.2.0:

  • Enhanced node types (Section, Table, Inline)
  • Path/CSS/XPath selection system
  • Additional format support (reStructuredText, HTML, Confluence)
  • Advanced tools (differ, validator, stats)
  • Interactive TUI

License

MIT - see LICENSE for details.