:py:mod:`cars.orchestrator.cluster.log_wrapper` =============================================== .. py:module:: cars.orchestrator.cluster.log_wrapper .. autoapi-nested-parse:: Contains functions for wrapper logs Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: cars.orchestrator.cluster.log_wrapper.AbstractLogWrapper cars.orchestrator.cluster.log_wrapper.LogWrapper cars.orchestrator.cluster.log_wrapper.CProfileWrapper cars.orchestrator.cluster.log_wrapper.MemrayWrapper cars.orchestrator.cluster.log_wrapper.CarsMemProf Functions ~~~~~~~~~ .. autoapisummary:: cars.orchestrator.cluster.log_wrapper.log_function cars.orchestrator.cluster.log_wrapper.time_profiling_function cars.orchestrator.cluster.log_wrapper.memory_profiling_function cars.orchestrator.cluster.log_wrapper.switch_messages cars.orchestrator.cluster.log_wrapper.log_message cars.orchestrator.cluster.log_wrapper.loop_function cars.orchestrator.cluster.log_wrapper.get_current_memory cars.orchestrator.cluster.log_wrapper.log_delta_memory cars.orchestrator.cluster.log_wrapper.generate_summary cars.orchestrator.cluster.log_wrapper.filter_lists cars.orchestrator.cluster.log_wrapper.generate_boxplot cars.orchestrator.cluster.log_wrapper.generate_histo cars.orchestrator.cluster.log_wrapper.generate_pie_chart cars.orchestrator.cluster.log_wrapper.cars_profile Attributes ~~~~~~~~~~ .. autoapisummary:: cars.orchestrator.cluster.log_wrapper.THREAD_TIMEOUT .. py:data:: THREAD_TIMEOUT :value: 2 .. py:class:: AbstractLogWrapper(conf_profiling, out_dir) AbstractLogWrapper .. py:attribute:: available_modes .. py:method:: register_subclass(short_name: str) :classmethod: Allows to register the subclass with its short name :param short_name: the subclass to be registered :type short_name: string .. py:method:: check_conf(conf) :abstractmethod: Check configuration :param conf: configuration to check :type conf: dict :return: overloaded configuration :rtype: dict .. py:method:: get_func_args_plus(func) :abstractmethod: getter for the args of the future function :param: function to apply :return: function to apply, overloaded key arguments .. py:class:: LogWrapper(conf_profiling, out_dir) Bases: :py:obj:`AbstractLogWrapper` LogWrapper simple log wrapper doing nothing .. py:method:: check_conf(conf) Check configuration :param conf: configuration to check :type conf: dict :return: overloaded configuration :rtype: dict .. py:method:: get_func_args_plus(func) getter for the args of the future function :param: function to apply :return: function to apply, overloaded key arguments .. py:class:: CProfileWrapper(conf_profiling, out_dir) Bases: :py:obj:`AbstractLogWrapper` CProfileWrapper log wrapper to analyze the internal time consuming of the function. The wrapper use cprofile API. .. py:method:: check_conf(conf) Check configuration :param conf: configuration to check :type conf: dict :return: overloaded configuration :rtype: dict .. py:method:: get_func_args_plus(func) getter for the args of the future function :param: function to apply :return: function to apply, overloaded key arguments .. py:class:: MemrayWrapper(conf_profiling, out_dir) Bases: :py:obj:`AbstractLogWrapper` MemrayWrapper log wrapper to analyze the internal allocation memory consuming of the function. The wrapper use cprofile API. .. py:method:: check_conf(conf) Check configuration :param conf: configuration to check :type conf: dict :return: overloaded configuration :rtype: dict .. py:method:: get_func_args_plus(func) getter for the args of the future function :param: function to apply :return: function to apply, overloaded key arguments .. py:function:: log_function(*argv, **kwargs) Create a wrapper for function running it :param argv: args of func :param kwargs: kwargs of func :return: path to results .. py:function:: time_profiling_function(*argv, **kwargs) Create a wrapper to profile the function elapse time :param argv: args of func :param kwargs: kwargs of func :return: path to results .. py:function:: memory_profiling_function(*argv, **kwargs) Create a wrapper to profile the function occupation memory :param argv: args of func :param kwargs: kwargs of func :return: path to results .. py:function:: switch_messages(func, total_time, max_memory=None) create profile message with specific message depends on elapsed time (LONG, FAST...). :param func : profiled function :param total_time : elapsed time of the function .. py:function:: log_message(func, message) log profiling message :param func : logged function :param message : log message .. py:function:: loop_function(argv, kwargs, func, nb_iteration=5) generate a loop on each cluster function to eval possible leak :param argv : input argv :param kwargs : input kwargs :param func : function to evaluation :param nb_iteration (int, optional): number of the iteration loop. :param Defaults to 5. Returns: _type_: result of the function .. py:function:: get_current_memory() Get current memory of process :return: memory :rtype: float .. py:function:: log_delta_memory(func, memory_start, memory_end) Log memory infos :param func: profiled function :param memory_start: memory before the run of function :type memory_start: float :param memory_end: memory after the run of function :type memory_end: float .. py:function:: generate_summary(out_dir, used_conf, clean_worker_logs=False) Generate Profiling summary .. py:function:: filter_lists(names, data, cond) Filter lists with condition on name .. py:function:: generate_boxplot(axis, names, data_full, title, data_type) Generate boxplot .. py:function:: generate_histo(axis, names, data, title, data_type, data_min_err=None, data_max_err=None) Generate histogram .. py:function:: generate_pie_chart(axis, names, data, title) Generate pie chart, data in % .. py:function:: cars_profile(name=None, interval=0.1) CARS profiling decorator :param: func: function to monitor .. py:class:: CarsMemProf(pid, pipe, interval=0.1) Bases: :py:obj:`threading.Thread` CarsMemProf Profiling thread .. py:method:: run() Run