abacusai.pipeline ================= .. py:module:: abacusai.pipeline Classes ------- .. autoapisummary:: abacusai.pipeline.Pipeline Module Contents --------------- .. py:class:: Pipeline(client, pipelineName=None, pipelineId=None, createdAt=None, notebookId=None, cron=None, nextRunTime=None, isProd=None, warning=None, createdBy=None, steps={}, pipelineReferences={}, latestPipelineVersion={}, codeSource={}, pipelineVariableMappings={}) Bases: :py:obj:`abacusai.return_class.AbstractApiClass` A Pipeline For Steps. :param client: An authenticated API Client instance :type client: ApiClient :param pipelineName: The name of the pipeline this step is a part of. :type pipelineName: str :param pipelineId: The reference to the pipeline this step belongs to. :type pipelineId: str :param createdAt: The date and time which the pipeline was created. :type createdAt: str :param notebookId: The reference to the notebook this pipeline belongs to. :type notebookId: str :param cron: A cron-style string that describes when this refresh policy is to be executed in UTC :type cron: str :param nextRunTime: The next time this pipeline will be run. :type nextRunTime: str :param isProd: Whether this pipeline is a production pipeline. :type isProd: bool :param warning: Warning message for possible errors that might occur if the pipeline is run. :type warning: str :param createdBy: The email of the user who created the pipeline :type createdBy: str :param steps: A list of the pipeline steps attached to the pipeline. :type steps: PipelineStep :param pipelineReferences: A list of references from the pipeline to other objects :type pipelineReferences: PipelineReference :param latestPipelineVersion: The latest version of the pipeline. :type latestPipelineVersion: PipelineVersion :param codeSource: information on the source code :type codeSource: CodeSource :param pipelineVariableMappings: A description of the function variables into the pipeline. :type pipelineVariableMappings: PythonFunctionArgument .. py:attribute:: pipeline_name :value: None .. py:attribute:: pipeline_id :value: None .. py:attribute:: created_at :value: None .. py:attribute:: notebook_id :value: None .. py:attribute:: cron :value: None .. py:attribute:: next_run_time :value: None .. py:attribute:: is_prod :value: None .. py:attribute:: warning :value: None .. py:attribute:: created_by :value: None .. py:attribute:: steps .. py:attribute:: pipeline_references .. py:attribute:: latest_pipeline_version .. py:attribute:: code_source .. py:attribute:: pipeline_variable_mappings .. py:attribute:: deprecated_keys .. py:method:: __repr__() .. py:method:: to_dict() Get a dict representation of the parameters in this class :returns: The dict value representation of the class parameters :rtype: dict .. py:method:: refresh() Calls describe and refreshes the current object's fields :returns: The current object :rtype: Pipeline .. py:method:: describe() Describes a given pipeline. :param pipeline_id: The ID of the pipeline to describe. :type pipeline_id: str :returns: An object describing a Pipeline :rtype: Pipeline .. py:method:: update(project_id = None, pipeline_variable_mappings = None, cron = None, is_prod = None) Updates a pipeline for executing multiple steps. :param project_id: A unique string identifier for the pipeline. :type project_id: str :param pipeline_variable_mappings: List of Python function arguments for the pipeline. :type pipeline_variable_mappings: List :param cron: A cron-like string specifying the frequency of the scheduled pipeline runs. :type cron: str :param is_prod: Whether the pipeline is a production pipeline or not. :type is_prod: bool :returns: An object that describes a Pipeline. :rtype: Pipeline .. py:method:: rename(pipeline_name) Renames a pipeline. :param pipeline_name: The new name of the pipeline. :type pipeline_name: str :returns: An object that describes a Pipeline. :rtype: Pipeline .. py:method:: delete() Deletes a pipeline. :param pipeline_id: The ID of the pipeline to delete. :type pipeline_id: str .. py:method:: list_versions(limit = 200) Lists the pipeline versions for a specified pipeline :param limit: The maximum number of pipeline versions to return. :type limit: int :returns: A list of pipeline versions. :rtype: list[PipelineVersion] .. py:method:: run(pipeline_variable_mappings = None) Runs a specified pipeline with the arguments provided. :param pipeline_variable_mappings: List of Python function arguments for the pipeline. :type pipeline_variable_mappings: List :returns: The object describing the pipeline :rtype: PipelineVersion .. py:method:: create_step(step_name, function_name = None, source_code = None, step_input_mappings = None, output_variable_mappings = None, step_dependencies = None, package_requirements = None, cpu_size = None, memory = None, timeout = None) Creates a step in a given pipeline. :param step_name: The name of the step. :type step_name: str :param function_name: The name of the Python function. :type function_name: str :param source_code: Contents of a valid Python source code file. The source code should contain the transform feature group functions. A list of allowed imports and system libraries for each language is specified in the user functions documentation section. :type source_code: str :param step_input_mappings: List of Python function arguments. :type step_input_mappings: List :param output_variable_mappings: List of Python function outputs. :type output_variable_mappings: List :param step_dependencies: List of step names this step depends on. :type step_dependencies: list :param package_requirements: List of package requirement strings. For example: ['numpy==1.2.3', 'pandas>=1.4.0']. :type package_requirements: list :param cpu_size: Size of the CPU for the step function. :type cpu_size: str :param memory: Memory (in GB) for the step function. :type memory: int :param timeout: Timeout for the step in minutes, default is 300 minutes. :type timeout: int :returns: Object describing the pipeline. :rtype: Pipeline .. py:method:: describe_step_by_name(step_name) Describes a pipeline step by the step name. :param step_name: The name of the step. :type step_name: str :returns: An object describing the pipeline step. :rtype: PipelineStep .. py:method:: unset_refresh_schedule() Deletes the refresh schedule for a given pipeline. :param pipeline_id: The id of the pipeline. :type pipeline_id: str :returns: Object describing the pipeline. :rtype: Pipeline .. py:method:: pause_refresh_schedule() Pauses the refresh schedule for a given pipeline. :param pipeline_id: The id of the pipeline. :type pipeline_id: str :returns: Object describing the pipeline. :rtype: Pipeline .. py:method:: resume_refresh_schedule() Resumes the refresh schedule for a given pipeline. :param pipeline_id: The id of the pipeline. :type pipeline_id: str :returns: Object describing the pipeline. :rtype: Pipeline .. py:method:: create_step_from_function(step_name, function, step_input_mappings = None, output_variable_mappings = None, step_dependencies = None, package_requirements = None, cpu_size = None, memory = None) Creates a step in the pipeline from a python function. :param step_name: The name of the step. :type step_name: str :param function: The python function. :type function: callable :param step_input_mappings: List of Python function arguments. :type step_input_mappings: List[PythonFunctionArguments] :param output_variable_mappings: List of Python function ouputs. :type output_variable_mappings: List[OutputVariableMapping] :param step_dependencies: List of step names this step depends on. :type step_dependencies: List[str] :param package_requirements: List of package requirement strings. For example: ['numpy==1.2.3', 'pandas>=1.4.0']. :type package_requirements: list :param cpu_size: Size of the CPU for the step function. :type cpu_size: str :param memory: Memory (in GB) for the step function. :type memory: int .. py:method:: wait_for_pipeline(timeout=1200) A waiting call until all the stages of the latest pipeline version is completed. :param timeout: The waiting time given to the call to finish, if it doesn't finish by the allocated time, the call is said to be timed out. :type timeout: int .. py:method:: get_status() Gets the status of the pipeline version. :returns: A string describing the status of a pipeline version (pending, running, complete, etc.). :rtype: str