Home

Overview

pymport allows you to transparently use Python libraries from Node.js. It contains a fully self-contained embedded Python interpreter. It can also be built to use an existing external Python installation.

Two two interpreters share the same main thread and memory space. Python objects and V8 objects remain separate. V8 can access Python objects which have a PyObject type, while Python cannot access V8 objects. Python objects referenced from JavaScript must be freed by the V8 GC before being marked as available from the Python GC.

When passing objects between the interpreters:

• JavaScript to Python conversion is automatic and by copying
• Python objects are passed by reference to JavaScript as a PyObject, refer to Using PyObjects directly and Using proxified PyObjects
• There is a loose types equivalence between Python and JavaScript allowing to transform variables, refer to Types equivalence, including the section of passing functions and handling exceptions
• Inline Python is also supported through pyval
• [New in 1.3] Read about asynchronous execution and multithreading in Asynchronous and background Python

Functions also can be freely passed between the two interpreters. Every time a cross-language function is called, it is executed by the corresponding interpreter. Their arguments will be converted according to the same rules.

Importing user modules from the current directory

pymport is made for using Python libraries in Node.js. When importing Python modules, by default pymport will search only the library paths.

In order to import a user module from the current directory, or any other user directory, PYTHONPATH must be set accordingly before initializing Python. In CommonJS this can be set before the require:

process.env['PYTHONPATH'] = _dirname;
const { pymport } = require('pymport');

In TypeScript or ES6, there is no easy way to do this - in this case PYTHONPATH should be set from the environment.

Known Issues

• In 1.0 the V8 GC does not take into account the memory held by a PyObjects when deciding if they should be GCed or when the heap limit has been reached
• In 1.1 and later the V8 GC takes into account the memory held by a PyObject when it is initially referenced in JS but not its eventual growth after being referenced
• In 1.0 Python objects of type function never expire, so you will be leaking memory if you create Python lambdas in a loop (fixed in 1.1)
• #3, PyObjects are leaking memory in synchronous loops
• #19, Python 3.11 and npm 6 (Node.js 14.x) does not support rebuilding from source
• #30], Using an external Python interpreter compiled against OpenSSL 3.0 on recent distributions (Ubuntu 22.04 for example) is not compatible with Node.js 16.x and results in a crash when loading modules that depend on SSL (numpy for example)
• #44], Cannot load built-in modules with binary code when using Python 3.11.1 on Windows

Supported Versions

pymport is unit-tested on all combinations of:

Platforms	Versions
OS	Windows 2019 & 2022, Ubuntu 20.04 & 22.04, macOS 11 & 12 (x86 only)
Node.js	14.x, 16.x and 18.x
Python	3.8, 3.9, 3.10 and 3.11

except for the following combinations that are not supported:

• Node.js 14.x with Python 3.11: rebuilding from source is not possible because of the node-gyp version which is too old to support Python 3.11

• Node.js 14.x/16.x on Ubuntu 22.04: rebuilding from source is not possible because Node.js has a built-in OpenSSL 1.1 while the system-provided Python is built vs OpenSSL 3.0

• Python 3.11.1 on Windows: An upstream bug renders this combination unusable #44]

Future Plans

• (longer term) Generate TypeScript bindings from the Python modules
• (longer term) Using Node.js packages from Python, ie, an eventual jimport project, is currently blocked by PR#4352