cars.orchestrator.cluster.log_wrapper

Contains functions for wrapper logs

Attributes

THREAD_TIMEOUT

Classes

CarsMemProf

CarsMemProf

Functions

`log_function`(argv, *kwargs)	Create a wrapper for function running it
`log_message`(func, message)	log profiling message
`loop_function`(argv, kwargs, func[, nb_iteration])	generate a loop on each cluster function to eval possible leak
`get_current_memory`()	Get current memory of process
`log_delta_memory`(func, memory_start, memory_end)	Log memory infos
`exception_safe`(func)	Decorator for consistent exception handling in profiling functions
`generate_summary`(out_dir, used_conf, pipeline_name[, ...])	Generate Profiling summary
`generate_pdf_profiling`(log_dir)	Generate PDF profiling summary for all res
`filter_lists`(names, data, cond)	Filter lists with condition on name
`generate_boxplot`(axis, names, data_full, title, data_type)	Generate boxplot
`generate_histo`(axis, names, data, title, data_type[, ...])	Generate histogram
`generate_pie_chart`(axis, names, data, title)	Generate pie chart, data in %
`cars_profile`([name, interval])	CARS profiling decorator

Module Contents

cars.orchestrator.cluster.log_wrapper.THREAD_TIMEOUT = 2

cars.orchestrator.cluster.log_wrapper.log_function(*argv, **kwargs)[source]

Create a wrapper for function running it

Parameters:

argv – args of func
kwargs – kwargs of func

Returns:

path to results

cars.orchestrator.cluster.log_wrapper.log_message(func, message)[source]

log profiling message

:param func : logged function :param message : log message

cars.orchestrator.cluster.log_wrapper.loop_function(argv, kwargs, func, nb_iteration=5)[source]

generate a loop on each cluster function to eval possible leak

:param argv : input argv :param kwargs : input kwargs :param func : function to evaluation :param nb_iteration (int, optional): number of the iteration loop. :param Defaults to 5.

Returns:: _type_: result of the function

cars.orchestrator.cluster.log_wrapper.get_current_memory()[source]

Get current memory of process

Returns:: memory
Return type:: float

cars.orchestrator.cluster.log_wrapper.log_delta_memory(func, memory_start, memory_end)[source]

Log memory infos

Parameters:

func – profiled function
memory_start (float) – memory before the run of function
memory_end (float) – memory after the run of function

cars.orchestrator.cluster.log_wrapper.exception_safe(func)[source]

Decorator for consistent exception handling in profiling functions

Parameters:: func – function to wrap
Returns:: wrapped function

cars.orchestrator.cluster.log_wrapper.generate_summary(out_dir, used_conf, pipeline_name, clean_worker_logs=False)[source]: Generate Profiling summary

cars.orchestrator.cluster.log_wrapper.generate_pdf_profiling(log_dir)[source]: Generate PDF profiling summary for all res

cars.orchestrator.cluster.log_wrapper.filter_lists(names, data, cond)[source]: Filter lists with condition on name

cars.orchestrator.cluster.log_wrapper.generate_boxplot(axis, names, data_full, title, data_type)[source]: Generate boxplot

cars.orchestrator.cluster.log_wrapper.generate_histo(axis, names, data, title, data_type, data_min_err=None, data_max_err=None)[source]: Generate histogram

cars.orchestrator.cluster.log_wrapper.generate_pie_chart(axis, names, data, title)[source]: Generate pie chart, data in %

cars.orchestrator.cluster.log_wrapper.cars_profile(name=None, interval=0.1)[source]

CARS profiling decorator

Param:: func: function to monitor

class cars.orchestrator.cluster.log_wrapper.CarsMemProf(pid, pipe, interval=0.1)[source]

Bases: threading.Thread

CarsMemProf

Profiling thread

pipe

interval = 0.1

cpu_interval = 0.1

process

run()[source]: Run