In this section, we will take a brief look at the internal organization of the Machin library, to better understand the functionality of every module.
machin.env has two sub modules:
machin.env.utils of the environment module provides
some convenient utility functions you might will need in your own application,
such as disabling the rendering window while keeping the rendered result in OpenAI gym.
machin.env.wrappers provides process-level parallel environment
wrappers for different environments.
machin.frame is the most important core part of the Machin library,
the framework module constitutes of:
machin.frame.algorithms: RL algorithm implementations.
machin.frame.buffers: Replay buffer implementations.
machin.frame.helpers: Utility to help you initialize a framework.
machin.frame.noise: Action space & parameter space noise generators.
machin.model is a collection of popular network models you might will use in your own
program, for example, ResNet.
Model module also contains the basis of all network modules:
this wrapper is built upon regular torch.nn.Module, and allows users to specify input/output
machin.parallel is the second core part of the Machin library,
the parallel module is a collection of refined implementations including:
machin.parallel.thread: Thread (With exception catching).
machin.parallel.process: Process (With remote exception catching).
machin.parallel.queues: Queues. (Used in pools).
machin.parallel.pool: Process pools (allow local functions,customize serialization policy), thread pools, pools with contexts, etc.
machin.parallel.assigner: Heuristic based model-device assignment.
machin.parallel.server: Implementations of different servers used in distributed
machin.parallel.distributed: A naive implementation of a part of
machin.utils is a messy hotchpotch of various tools, it is very hard to categorize them,
but they could be helpful sometimes, so we left them here:
machin.utils.checker: A checker implementation, using forward & backward hooksprovided by pytorch to check the input/ouput, input gradient of models. Supports userdefined checkers and tensorboard.
machin.utils.conf: Functions designed to load/save a json configuration file, aswell as loading parametrs from commandline.
machin.utils.helper_classes: Various helper classes, such as
machin.utils.learning_rate: Functions used in learning rate schedulers. Usefulif you would like to have finer control over the learning rate.
machin.utils.loading: Logging utility module.
machin.utils.media: Media writing utility, mainly images and videos, useful if you wouldlike to log rendered environments.
machin.utils.prepare: Functions used to create directories, loading models (take care ofdevices automatically), for preparing a training session.
machin.utils.save_env: A standard reinforcement training environment creator, will createunique directories by time for you.
machin.utils.visualize: Visualize your model, currently only contains some simple functionsfor gradient flow checking.
machin.utils.tensorboard: A simple tensorboard wrapper.