Skip to content

PyLGEN: The Python-Native Compiler Framework for Building Interpreters and DSLs

GitHub Repo PyPI 0.7.2 PyPI - 3.12 PyPI - LICENSE CI codecov Supported OS

A Python-native compiler framework that brings the "Dragon Book" to life in Python with clarity throughout. A high-performance tool to build interpreters and Domain-Specific Languages (DSLs). Master every step of the compilation pipeline, from lexing to semantic analysis, with full control and clarity.

  • Build interpreters and compilers from scratch, without leaving the Python's ecosystem
  • Keep total control of what's going on at every step
  • Build fast and easy with python for prototyping and debugging
  • Compile and get more speed with cython

What is PyLGEN?

PyLGEN is a toolbox for building interpreters and compilers from scratch, without leaving the Python ecosystem. It's not a magical code generator, it's a cohesive set of modules that guide you through every stage: from lexical analysis to semantic evaluation, including LALR(1) parsing and Abstract Syntax Tree (AST) construction.

TL;DR: PyLGEN is a powerful Python-native toolkit for creating fast interpreters, compilers, and DSLs. It gives you complete control over the compilation process, from lexical analysis to AST evaluation, and is optimized with Cython for near-C performance. Ideal for students, software engineers, and anyone exploring compiler theory. Explore the complete source code, open issues, and contribute on GitHub.

Why PyLGEN?

What makes PyLGEN different from other parser generators or compiler frameworks?

  • Prototype fast, optimize smart: Write your entire language logic in pure python first. Only when performance becomes critical, compile your critical path with Cython.
  • Full visibility, zero magic: See exactly what happens at each stage. You build your own ASTs, define your own visitors, and control traversal order. There are no hidden transformations, you own the pipeline.
  • Parsing: an LALR(1) engine with conflict detection and panic-mode error recovery. It lets your DSL process malformed input without stopping.
  • Instant visual feedback: Turn your automata, ASTs and parse trees into interactive HTML graphs with a single command. Debug visually, not just via logs, a game changer for understanding and teaching.
  • Feels like Python: No weird configuration files, no custom DSLs for your grammar. Just Python code, all the way down. Use your favorite libraries, test with pytest, and deploy as a standard package.

Philosophy: Clarity and control

Building a language should be fun, educational and productive. That's why PyLGEN focuses on:

  • Transparency: No black boxes. Every step, from token definitions to AST visitation, is under your control.
  • Python native: Write your logic in pure Python, leverage its ecosystem, and debug just as you always do.
  • Performance: The framework core is optimized in Cython. You can compile your own Cython extensions if you need more performance.

Who is it for?

  • Students and teachers: PyLGEN follows the classic compiler course flow, but with hands-on approach.
  • Software engineers: For creating DSLs, configuration languages, or code analysis tools without being locked into monolithic generators.
  • Curious explorers: For experimenting and understanding how a compiler works under the hood.

Intention

PyLGEN has the intention to be the meeting point between classic compiler theory and modern software engineering practice. It's not trying to be the only framework or the fastest in every case, but to be the one that lets you understand, modify, and optimize every piece with confidence.

Where to Go Next?

  • Quick Start Guide: Get started building your first interpreter in minutes.
  • Benchmark: PyLGEN vs Lark: See how PyLGEN outperforms other tools in real-world tests.
  • Architecture Deep Dive: Understand the modular design and each component of the framework.
  • Under the hood: Build a lexer and parser from scratch, using the same building blocks that the framework uses internally.
  • Examples: See practical examples of using PyLGEN to build interpreters and DSLs.
  • Table of Contents: Browse the full documentation, organized by topic.
  • GitHub Repository: Star the project, report issues, and explore the source code.