Mars is an open-sourced project released under Apache License 2.0. We welcome and thanks for your contributions. Here are some guidelines you may find useful when you want to make some change of Mars.
Mars hosts and maintains its code on Github. We provide a generalized guide for opening issues and pull requests.
Unless you want to develop or debug Mars under Windows, it is strongly recommended to develop Mars under MacOS or Linux, where you can test all functions of Mars. The steps listed below are applicable on MacOS and Linux.
It is recommended to develop Mars with conda. When you want ot install Mars for development, use the following steps to create an environment and install Mars inside it:
git clone https://github.com/mars-project/mars.git cd mars conda create -n mars-dev --file conda-spec.txt python=3.7 source activate mars-dev conda install -c conda-forge pyarrow tiledb-py pip install -e ".[dev]"
Change 3.7 into the version of Python you want to install, and mars-dev into your preferred environment name.
3.7
mars-dev
Mars has a dev option for installation. When you want to install Mars for development, use the following steps:
dev
pip install --upgrade setuptools pip pip install cython protobuf git clone https://github.com/mars-project/mars.git cd mars pip install -e ".[dev]"
If you are using a system-wide Python installation and you only want to install Mars for you, you can add --user to the pip install commands.
--user
pip install
After installing Mars, you can check if Mars is installed correctly by running
python -c "import mars; print(mars.__version__)"
If this command correctly outputs your Mars version, Mars is correctly installed.
Mars uses Cython to accelerate part of its code. After you change Cython source code, you need to compile them into binaries by executing the command below on the root of Mars project:
python setup.py build_ext -i
Mars uses Protobuf to serialize operands internally. After you change protobuf files in Mars, you need to compile them into Python source codes by running the command below on the root of Mars project:
python setup.py build_proto
Note that besides files compiled by protoc, Mars will also generate an opcodes.py given operand.proto. You shall not edit the generated file.
protoc
opcodes.py
operand.proto
Mars will download protoc for Win32, MacOS x64 and Linux x64 from Github if protoc cannot be found. If you need to specify a customized version of protoc, you can use the environment variable PROTOC to specify its path before calling the above command.
PROTOC
It is recommended to use pytest to run Mars tests. A simple command below will run all the tests of Mars:
pytest
pytest mars
If you want to generate a coverage report as well, you can run:
pytest --cov=mars --cov-report=html mars
Coverage report will be put into the directory htmlcov.
htmlcov
The command above does not contain coverage data for Cython files by default. To obtain coverage data about Cython files, you can run
CYTHON_TRACE=1 python setup.py build_ext -i --force
before running the pytest command mentioned above. After report is generated, it it recommended to remove all generated C files and binaries and rebuild without CYTHON_TRACE, as this option will reduce the performance of Mars.
CYTHON_TRACE
Mars uses sphinx to build documents. You need to install necessary packages with the command below to install these dependencies and build your documents into HTML.
sphinx
pip install -r docs/requirements-doc.txt cd docs make html
The built documents are in docs/build/html directory.
docs/build/html
When you want to create translations of Mars documents, you may append -l <locale> after the I18NSPHINXLANGS variable in Makefile. Currently only simplified Chinese is supported. After that, run the command below to generate portable files (*.po) for the documents, which are in docs/source/locale/<locale>/LC_MESSAGES:
-l <locale>
I18NSPHINXLANGS
Makefile
*.po
docs/source/locale/<locale>/LC_MESSAGES
cd docs make gettext
After that you can translate Mars documents into your language. Note that when you run make gettext again, translations will be broken into a fixed-width text. For Chinese translators, you need to install jieba to get this effect.
make gettext
jieba
When you finish translation, you can run
cd docs # change LANG into the language you want to build make -e SPHINXOPTS="-D language='LANG'" html
to build the document in the language you just translated into.