ML之catboost：catboost模型中常用的Pool类型数据结构源代码解读、案例应用之详细攻略

ML之catboost：catboost模型中常用的Pool类型数据结构源代码解读、案例应用之详细攻略catboost模型中常用的Pool类型数据源结构代码解读Pool简介Pool 是在CatBoost中用作训练模型的数据结构。Pool的案例应用train_dataPool = Pool(data=[[12, 14, 16, 18], [23, 25, 27, 29], [32, 34, 36, 38]], label=[10, 20, 30], weight=[0.1, 0.2, 0.3]) # weight各自权重默认设置为1 Pool源代码解读class Pool Found at: catboost.coreclass Pool(_PoolBase):"""Pool used in CatBoost as a data structure to train model from."""def __init__(self,data,label=None,cat_features=None,text_features=None,embedding_features=None,column_description=None,pairs=None,delimiter='\t',has_header=False,ignore_csv_quoting=False,weight=None,group_id=None,group_weight=None,subgroup_id=None,pairs_weight=None,baseline=None,feature_names=None,thread_count=-1):"""Pool is an internal data structure that is used by CatBoost.You can construct Pool from list, numpy.ndarray, pandas.DataFrame, pandas.Series.Parameters----------Pool 是在CatBoost中用作训练模型的数据结构。data : list or numpy.ndarray or pandas.DataFrame or  pandas.Series or FeaturesData or string. Data source of Pool. If list or numpy.ndarrays or pandas.DataFrame or pandas.  Series, giving 2 dimensional array like data.  If FeaturesData - see FeaturesData description for  details, 'cat_features' and 'feature_names' parameters must be equal to None in this case. If string, giving the path to the file with data in catboost  format. If path starts with "quantized://", the file has to contain  quantized dataset saved with Pool.save().label : list or numpy.ndarrays or pandas.DataFrame or  pandas.Series, optional (default=None). Label of the training data. If not None, giving 1 or 2 dimensional array like data   with floats. If data is a file, then label must be in the file, that is label must be equals to None  cat_features : list or numpy.ndarray, optional  (default=None). If not None, giving the list of Categ features indices or names. If it contains feature names, Pool's feature names must be defined: either by passing 'feature_names' parameter or if data is pandas.DataFrame (feature  names are initialized from it's column names). Must be None if 'data' parameter has FeaturesData typetext_features : list or numpy.ndarray, optional . (default=None), If not None, giving the list of Text features indices or  names. If it contains feature names, Pool's feature names mustbe defined: either by passing 'feature_names'  parameter or if data is pandas.DataFrame (feature names are initialized from it's column names).  Must be None if 'data' parameter has FeaturesData typeembedding_features : list or numpy.ndarray, optional . (default=None). If not None, giving the list of Embedding features  indices or names. If it contains feature names, Pool's feature names must  be defined: either by passing 'feature_names' parameter or if data is pandas.DataFrame (feature names are initialized from it's column names). Must be None if 'data' parameter has FeaturesData typecolumn_description : string, optional (default=None). ColumnsDescription parameter. There are several columns description types: Label,   Categ, Num, Auxiliary, DocId, Weight, Baseline, GroupId,  Timestamp.  All columns are Num as default, it's not necessary to   specify  this type of columns. Default Label column index is 0   (zero).  If None, Label column is 0 (zero) as default, all data   columns are Num as default. If string, giving the path to the file with  ColumnsDescription in column_description format.pairs : list or numpy.ndarray or pandas.DataFrame or string.The pairs description.  If list or numpy.ndarrays or pandas.DataFrame, giving 2  dimensional. The shape should be Nx2, where N is the pairs' count.  The first element of the pair is  the index of winner object in the training set. The  second element of the pair is the index of loser object in the training set. If string, giving the path to the file with pairs description.delimiter : string, optional (default='\t'). Delimiter to use for separate features in file. Should be only one symbol, otherwise would be taken  only the first character of the string.has_header : bool optional (default=False). If True, read column names from first line.ignore_csv_quoting : bool optional (default=False). If True ignore quoting '"'.weight : list or numpy.ndarray, optional (default=None). Weight for each instance. If not None, giving 1 dimensional array like data.group_id : list or numpy.ndarray, optional (default=None). group id for each instance.  If not None, giving 1 dimensional array like data.group_weight : list or numpy.ndarray, optional .(default=None).Group weight for each instance.  If not None, giving 1 dimensional array like data.subgroup_id : list or numpy.ndarray, optional . (default=None) .subgroup id for each instance. If not None, giving 1 dimensional array like data.pairs_weight : list or numpy.ndarray, optional .(default=None).  Weight for each pair. If not None, giving 1 dimensional array like pairs.baseline : list or numpy.ndarray, optional (default=None).Baseline for each instance. If not None, giving 2 dimensional array like data.feature_names : list or string, optional (default=None). If list - list of names for each given data_feature. If string - path with scheme for feature names data to  load. If this parameter is None and 'data' is pandas.DataFrame   feature names will be initialized  from DataFrame's column names. Must be None if 'data' parameter has FeaturesData typethread_count : int, optional (default=-1).Thread count for data processing. If -1, then the number of threads is set to the number of  CPU cores.data : 列表或numpy。ndarray或pandas.DataFrame或pandas.Series或特性数据或字符串。池的数据源。如果列表或numpy。ndarrays或pandas.DataFrame或pandas.Series，给出类似数据的二维数组。如果FeaturesData -参见FeaturesData描述的详细信息，在这种情况下'cat_features'和'feature_names'参数必须等于None。如果字符串，以catboost格式提供文件的路径。如果path以"quantized://"开头，则文件必须包含保存在Pool.save()中的量化数据集。label : 列表或numpy。ndarrays或pandas.DataFrame或pandas.Series,可选(默认= None)。训练数据的标签。如果不是None，则给出1维或2维的数组，如带有浮点数的data。如果data是一个文件，那么label必须在文件中，即label必须等于None cat_features: list或numpy。ndarray,可选(默认= None)。如果不是None，则给出类别特征索引或名称的列表。如果包含特性名称，则必须定义池的特性名称:通过传递'feature_names'参数或如果data是pandas。DataFrame(特性名称从它的列名初始化)。如果“data”参数具有FeaturesData类型，必须为Nonetext_features : 列表或numpy。ndarray,可选的。(default=None)，如果不是None，则给出文本特性的索引或名称。如果它包含特性名称，则池的特性名称必须be defined: 通过传递'feature_names'参数或如果数据是pandas.DataFrame特性名称从它的列名初始化)。如果“data”参数具有FeaturesData类型，必须为Noneembedding_features : list或numpy。ndarray,可选的。(默认= None)。如果不是None，则给出嵌入特性的索引或名称。如果包含特性名称，则必须定义池的特性名称:通过传递'feature_names'参数或如果data是pandas。DataFrame(特性名称从它的列名初始化)。如果“data”参数具有FeaturesData类型，必须为Nonecolumn_description : 字符串，可选(默认为None)。ColumnsDescription参数。有几个列描述类型:Label, Categ, Num, Auxiliary, DocId, Weight, Baseline, GroupId, Timestamp。默认情况下，所有列都是Num，没有必要指定这种类型的列。默认标签列索引为0(零)。如果为None，则Label column默认为0(0)，所有数据列默认为Num。如果是字符串，则以column_description格式给出该文件的路径。pairs : 列表或numpy。ndarray或pandas.DataFrame或字符串。对描述。如果列表或numpy。ndarrays或pandas.DataFrame，给出二维。形状应该是Nx2，其中N是对的计数。pair的第一个元素是训练集中赢家对象的索引。对的第二个元素是输家对象在训练集中的索引。如果是字符串，则给出文件的路径和对的描述。delimiter : 字符串，可选(默认='\t')。用于文件中单独特性的分隔符。应该只有一个符号，否则将只取字符串的第一个字符。has_header : bool可选(默认为False)。如果为True，则从第一行读取列名。ignore_csv_quoting :bool可选(默认为False)。如果为真，请忽略“”。weight :列表或numpy。ndarray,可选(默认= None)。每个实例的权重。如果不是None，则给出类似data的一维数组。group_id : list或numpy。ndarray,可选(默认= None)。每个实例的组id。如果不是None，则给出类似data的一维数组。group_weight : list或numpy。ndarray,可选的。(默认= None)。每个实例的组权重。如果不是None，则给出类似data的一维数组。subgroup_id : list或numpy。ndarray,可选的。(default=None) .subgroup每个实例的id。如果不是None，则给出类似data的一维数组。pairs_weight :列表或numpy。ndarray,可选的。(默认= None)。每双的重量。如果不是None，则给出一个一维数组。baseline：列表或numpy。ndarray,可选(默认= None)。每个实例的基线。如果不是None，则给出像data这样的二维数组。feature_names : 列表或字符串，可选(默认为None)。If list -每个给定data_feature的名称列表。If string - path with scheme for feature names要加载的数据。如果此参数为None且'data'为pandas.DataFrame特性名称将从DataFrame的列名初始化。如果“data”参数具有FeaturesData类型，必须为Nonethread_count : nt，可选(默认=-1)。用于数据处理的线程计数。如果-1，则线程数设置为CPU核数。if data is not None:self._check_data_type(data)self._check_data_empty(data)if pairs is not None and isinstance(data, STRING_TYPES) != isinstance(pairs, STRING_TYPES):raise CatBoostError("data and pairs parameters shouldbe the same types.")if column_description is not None and not isinstance(data, STRING_TYPES):raise CatBoostError("data should be the string type ifcolumn_description parameter is specified.")if isinstance(data, STRING_TYPES):if any(v is not None for v in [cat_features,text_features, embedding_features, weight, group_id,group_weight,subgroup_id, pairs_weight, baseline, label]):raise CatBoostError("cat_features, text_features, embedding_features,weight, group_id, group_weight, subgroup_id, pairs_weight, ""baseline, label should have the None type whenthe pool is read from the file.")if (feature_names is not None) and (not isinstance(feature_names, STRING_TYPES)):raise CatBoostError("feature_names should have None or string typewhen the pool is read from the file.")self._read(data, column_description, pairs,feature_names, delimiter, has_header, ignore_csv_quoting,thread_count)else:if isinstance(data, FeaturesData):if any(v is not None for v in [cat_features,text_features, embedding_features, feature_names]):raise CatBoostError("cat_features, text_features,embedding_features, feature_names should have the Nonetype"" when 'data' parameter has FeaturesData type")elif isinstance(data, np.ndarray):if (data.dtype.kind == 'f') and (cat_features is notNone) and (len(cat_features) > 0):raise CatBoostError("'data' is numpy array offloating point numerical type, it means no categorical features,"" but 'cat_features' parameter specifies nonzeronumber of categorical features")if (data.dtype.kind == 'f') and (text_features is notNone) and (len(text_features) > 0):raise CatBoostError("'data' is numpy array of floating pointnumerical type, it means no text features,"" but 'text_features' parameter specifiesnonzero number of text features")if (data.dtype.kind != 'O') and (embedding_featuresis not None) and (len(embedding_features) > 0):raise CatBoostError("'data' is numpy array of non-object type, itmeans no embedding features,"" but 'embedding_features' parameter specifiesnonzero number of embedding features")elif isinstance(data, scipy.sparse.spmatrix):if (data.dtype.kind == 'f') and (cat_features is notNone) and (len(cat_features) > 0):raise CatBoostError("'data' is scipy.sparse.spmatrix of floating point numerical type, it means nocategorical features,"" but 'cat_features' parameter specifies nonzeronumber of categorical features")if (text_features is not None) and (len(text_features)> 0):raise CatBoostError("'data' is scipy.sparse.spmatrix, it means no textfeatures,"" but 'text_features' parameter specifiesnonzero number of text features")if (embedding_features is not None) and (len(embedding_features) > 0):raise CatBoostError("'data' is scipy.sparse.spmatrix, it means noembedding features,"" but 'embedding_features' parameter specifiesnonzero number of embedding features")if isinstance(feature_names, STRING_TYPES):raise CatBoostError("feature_names must be None or have non-stringtype when the pool is created from ""python objects.")self._init(data, label, cat_features, text_features,embedding_features, pairs, weight, group_id, group_weight,subgroup_id, pairs_weight, baseline, feature_names,thread_count)super(Pool, self).__init__()def _check_files(self, data, column_description, pairs):"""Check files existence."""if data.find('://') == -1 and not os.path.isfile(data):raise CatBoostError("Invalid data path='{}': file does notexist.".format(data))if column_description is not None and column_description.find('://') == -1 and not os.path.isfile(column_description):raise CatBoostError("Invalid column_descriptionpath='{}': file does not exist.".format(column_description))if pairs is not None and pairs.find('://') == -1 and not os.path.isfile(pairs):raise CatBoostError("Invalid pairs path='{}': file does notexist.".format(pairs))def _check_delimiter(self, delimiter):if not isinstance(delimiter, STRING_TYPES):raise CatBoostError("Invalid delimiter type={} : must bestr().".format(type(delimiter)))if len(delimiter) < 1:raise CatBoostError("Invalid delimiter length={} : mustbe > 0.".format(len(delimiter)))def _check_column_description_type(self,column_description):"""Check type of column_description parameter."""if not isinstance(column_description, STRING_TYPES):raise CatBoostError("Invalid column_description type={}:must be str().".format(type(column_description)))def _check_string_feature_type(self, features, features_name):"""Check type of cat_feature parameter."""if not isinstance(features, (list, np.ndarray)):raise CatBoostError("Invalid {} type={}: must be list() ornp.ndarray().".format(features_name, type(features)))def _check_string_feature_value(self, features,features_count, features_name):"""Check values in cat_feature parameter. Must be int indices."""for indx, feature in enumerate(features):if not isinstance(feature, INTEGER_TYPES):raise CatBoostError("Invalid {}[{}] = {} value type={}:must be int().".format(features_name, indx, feature, type(feature)))if feature >= features_count:raise CatBoostError("Invalid {}[{}] = {} value: index mustbe < {}.".format(features_name, indx, feature, features_count))def _check_pairs_type(self, pairs):"""Check type of pairs parameter."""if not isinstance(pairs, (list, np.ndarray, DataFrame)):raise CatBoostError("Invalid pairs type={}: must be list(),np.ndarray() or pd.DataFrame.".format(type(pairs)))def _check_pairs_value(self, pairs):"""Check values in pairs parameter. Must be int indices."""for pair_id, pair in enumerate(pairs):if (len(pair) != 2):raise CatBoostError("Length of pairs[{}] isn't equal to2.".format(pair_id))for i, index in enumerate(pair):if not isinstance(index, INTEGER_TYPES):raise CatBoostError("Invalid pairs[{}][{}] = {} valuetype={}: must be int().".format(pair_id, i, index, type(index)))def _check_data_type(self, data):"""Check type of data."""if not isinstance(data, (STRING_TYPES, ARRAY_TYPES,SPARSE_MATRIX_TYPES, FeaturesData)):raise CatBoostError("Invalid data type={}: data must be list(), np.ndarray(),DataFrame(), Series(), FeaturesData " + " scipy.sparse matrix orfilename str().".format(type(data)))def _check_data_empty(self, data):"""Check that data is not empty (0 objects is ok).note: already checked if data is FeatureType, so no needto check again"""if isinstance(data, STRING_TYPES):if not data:raise CatBoostError("Features filename is empty.")elif isinstance(data, (ARRAY_TYPES,SPARSE_MATRIX_TYPES)):data_shape = np.shape(data)if len(data_shape) == 1 and data_shape[0] > 0:if isinstance(data[0], Iterable):data_shape = tuple(data_shape + tuple([len(data[0])]))else:data_shape = tuple(data_shape + tuple([1]))if not len(data_shape) == 2:raise CatBoostError("Input data has invalid shape: {}.Must be 2 dimensional".format(data_shape))if data_shape[1] == 0:raise CatBoostError("Input data must have at least onefeature")def _check_label_type(self, label):"""Check type of label."""if not isinstance(label, ARRAY_TYPES):raise CatBoostError("Invalid label type={}: must be arraylike.".format(type(label)))def _check_label_empty(self, label):"""Check label is not empty."""if len(label) == 0:raise CatBoostError("Labels variable is empty.")def _check_label_shape(self, label, samples_count):"""Check label length and dimension."""if len(label) != samples_count:raise CatBoostError("Length of label={} and length ofdata={} is different.".format(len(label), samples_count))def _check_baseline_type(self, baseline):"""Check type of baseline parameter."""if not isinstance(baseline, ARRAY_TYPES):raise CatBoostError("Invalid baseline type={}: must bearray like.".format(type(baseline)))def _check_baseline_shape(self, baseline, samples_count):"""Check baseline length and dimension."""if len(baseline) != samples_count:raise CatBoostError("Length of baseline={} and length ofdata={} are different.".format(len(baseline), samples_count))if not isinstance(baseline[0], Iterable) or isinstance(baseline[0], STRING_TYPES):raise CatBoostError("Baseline must be 2 dimensionaldata, 1 column for each class.")try:if np.array(baseline).dtype not in (np.dtype('float'), np.dtype('float32'), np.dtype('int')):raise CatBoostError()except CatBoostError:raise CatBoostError("Invalid baseline value type={}: mustbe float or int.".format(np.array(baseline).dtype))def _check_weight_type(self, weight):"""Check type of weight parameter."""if not isinstance(weight, ARRAY_TYPES):raise CatBoostError("Invalid weight type={}: must bearray like.".format(type(weight)))def _check_weight_shape(self, weight, samples_count):"""Check weight length."""if len(weight) != samples_count:raise CatBoostError("Length of weight={} and length ofdata={} are different.".format(len(weight), samples_count))if not isinstance(weight[0], (INTEGER_TYPES,FLOAT_TYPES)):raise CatBoostError("Invalid weight value type={}: mustbe 1 dimensional data with int, float or long types.".format(type(weight[0])))def _check_group_id_type(self, group_id):"""Check type of group_id parameter."""if not isinstance(group_id, ARRAY_TYPES):raise CatBoostError("Invalid group_id type={}: must bearray like.".format(type(group_id)))def _check_group_id_shape(self, group_id, samples_count):"""Check group_id length."""if len(group_id) != samples_count:raise CatBoostError("Length of group_id={} and lengthof data={} are different.".format(len(group_id), samples_count))def _check_group_weight_type(self, group_weight):"""Check type of group_weight parameter."""if not isinstance(group_weight, ARRAY_TYPES):raise CatBoostError("Invalid group_weight type={}: mustbe array like.".format(type(group_weight)))def _check_group_weight_shape(self, group_weight,samples_count):"""Check group_weight length."""if len(group_weight) != samples_count:raise CatBoostError("Length of group_weight={} andlength of data={} are different.".format(len(group_weight),samples_count))if not isinstance(group_weight[0], (FLOAT_TYPES)):raise CatBoostError("Invalid group_weight value type={}:must be 1 dimensional data with float types.".format(type(group_weight[0])))def _check_subgroup_id_type(self, subgroup_id):"""Check type of subgroup_id parameter."""if not isinstance(subgroup_id, ARRAY_TYPES):raise CatBoostError("Invalid subgroup_id type={}: mustbe array like.".format(type(subgroup_id)))def _check_subgroup_id_shape(self, subgroup_id,samples_count):"""Check subgroup_id length."""if len(subgroup_id) != samples_count:raise CatBoostError("Length of subgroup_id={} andlength of data={} are different.".format(len(subgroup_id),samples_count))def _check_feature_names(self, feature_names,num_col=None):if num_col is None:num_col = self.num_col()if not isinstance(feature_names, Sequence):raise CatBoostError("Invalid feature_names type={} :must be list".format(type(feature_names)))if len(feature_names) != num_col:raise CatBoostError("Invalid length of feature_names={} :must be equal to the number of columns in data={}".format(len(feature_names), num_col))def _check_thread_count(self, thread_count):if not isinstance(thread_count, INTEGER_TYPES):raise CatBoostError("Invalid thread_count type={} : mustbe int".format(type(thread_count)))def slice(self, rindex):if not isinstance(rindex, ARRAY_TYPES):raise CatBoostError("Invalid rindex type={} : must be listor numpy.ndarray".format(type(rindex)))slicedPool = Pool(None)slicedPool._take_slice(self, rindex)return slicedPooldef set_pairs(self, pairs):self._check_pairs_type(pairs)if isinstance(pairs, DataFrame):pairs = pairs.valuesself._check_pairs_value(pairs)self._set_pairs(pairs)return selfdef set_feature_names(self, feature_names):self._check_feature_names(feature_names)self._set_feature_names(feature_names)return selfdef set_baseline(self, baseline):self._check_baseline_type(baseline)baseline = self._if_pandas_to_numpy(baseline)baseline = np.reshape(baseline, (self.num_row(), -1))self._check_baseline_shape(baseline, self.num_row())self._set_baseline(baseline)return selfdef set_weight(self, weight):self._check_weight_type(weight)weight = self._if_pandas_to_numpy(weight)self._check_weight_shape(weight, self.num_row())self._set_weight(weight)return selfdef set_group_id(self, group_id):self._check_group_id_type(group_id)group_id = self._if_pandas_to_numpy(group_id)self._check_group_id_shape(group_id, self.num_row())self._set_group_id(group_id)return selfdef set_group_weight(self, group_weight):self._check_group_weight_type(group_weight)group_weight = self._if_pandas_to_numpy(group_weight)self._check_group_weight_shape(group_weight, self.num_row())self._set_group_weight(group_weight)return selfdef set_subgroup_id(self, subgroup_id):self._check_subgroup_id_type(subgroup_id)subgroup_id = self._if_pandas_to_numpy(subgroup_id)self._check_subgroup_id_shape(subgroup_id, self.num_row())self._set_subgroup_id(subgroup_id)return selfdef set_pairs_weight(self, pairs_weight):self._check_weight_type(pairs_weight)pairs_weight = self._if_pandas_to_numpy(pairs_weight)self._check_weight_shape(pairs_weight, self.num_pairs())self._set_pairs_weight(pairs_weight)return selfdef save(self, fname):"""Save the quantized pool to a file.        Parameters----------fname : stringOutput file name."""if not self.is_quantized():raise CatBoostError('Pool is not quantized')if not isinstance(fname, STRING_TYPES):raise CatBoostError("Invalid fname type={}: must bestr().".format(type(fname)))self._save(fname)def quantize(self, ignored_features=None,per_float_feature_quantization=None, border_count=None,max_bin=None, feature_border_type=None,sparse_features_conflict_fraction=None,nan_mode=None, input_borders=None, task_type=None,used_ram_limit=None, random_seed=None, **kwargs):"""Quantize this poolParameters----------pool : catboost.PoolDataset to quantize.ignored_features : list, [default=None]Indices or names of features that should be excludedwhen training.per_float_feature_quantization : list of strings,[default=None]List of float binarization descriptions.Format : described in documentation on catboost.aiExample 1: ['0:1024'] means that feature 0 will have 1024borders.Example 2: ['0:border_count=1024', '1:border_count=1024', ...] means that two first features have1024 borders.Example 3: ['0:nan_mode=Forbidden,border_count=32,border_type=GreedyLogSum','1:nan_mode=Forbidden,border_count=32,border_type=GreedyLogSum'] - defines more quantizationproperties for first two features.border_count : int, [default = 254 for training on CPU or128 for training on GPU]The number of partitions in numeric featuresbinarization. Used in the preliminary calculation.range: [1,65535] on CPU, [1,255] on GPUmax_bin : float, synonym for border_count.feature_border_type : string, [default='GreedyLogSum']The binarization mode in numeric features binarization.Used in the preliminary calculation.Possible values:- 'Median'- 'Uniform'- 'UniformAndQuantiles'- 'GreedyLogSum'- 'MaxLogSum'- 'MinEntropy'sparse_features_conflict_fraction : float, [default=0.0]CPU only. Maximum allowed fraction of conflicting non-default values for features in exclusive features bundle.Should be a real value in [0, 1) interval.nan_mode : string, [default=None]Way to process missing values for numeric features.Possible values:- 'Forbidden' - raises an exception if there is a missingvalue for a numeric feature in a dataset.- 'Min' - each missing value will be processed as theminimum numerical value.- 'Max' - each missing value will be processed as themaximum numerical value.If None, then nan_mode=Min.input_borders : string, [default=None]input file with borders used in numeric featuresbinarization.task_type : string, [default=None]The calcer type used to train the model.Possible values:- 'CPU'- 'GPU'used_ram_limit=Nonerandom_seed : int, [default=None]The random seed used for data sampling.If None, 0 is used."""if self.is_quantized():raise CatBoostError('Pool is already quantized')params = {}_process_synonyms(params)if border_count is None:border_count = max_bindev_efb_max_buckets = kwargs.pop('dev_efb_max_buckets', None)dev_max_subset_size_for_build_borders = kwargs.pop('dev_max_subset_size_for_build_borders', None)if kwargs:raise CatBoostError("got an unexpected keywordarguments: {}".format(kwargs.keys()))_update_params_quantize_part(params, ignored_features,per_float_feature_quantization, border_count,feature_border_type, sparse_features_conflict_fraction,dev_efb_max_buckets, nan_mode, input_borders, task_type,used_ram_limit, random_seed,dev_max_subset_size_for_build_borders)self._quantize(params)def _if_pandas_to_numpy(self, array):if isinstance(array, Series):array = array.valuesif isinstance(array, DataFrame):array = np.transpose(array.values)[0]return arraydef _label_if_pandas_to_numpy(self, label):if isinstance(label, Series):label = label.valuesif isinstance(label, DataFrame):label = label.valuesreturn labeldef _read(self,pool_file,column_description,pairs,feature_names_path,delimiter,has_header,ignore_csv_quoting,thread_count,quantization_params=None):"""Read Pool from file."""with log_fixup():self._check_files(pool_file, column_description, pairs)self._check_delimiter(delimiter)if column_description is None:column_description = ''else:self._check_column_description_type(column_description)if pairs is None:pairs = ''if feature_names_path is None:feature_names_path = ''self._check_thread_count(thread_count)self._read_pool(pool_file, column_description, pairs,feature_names_path, delimiter[0], has_header,ignore_csv_quoting, thread_count, quantization_params)def _init(self,data,label,cat_features,text_features,embedding_features,pairs, weight,group_id,group_weight,subgroup_id,pairs_weight,baseline,feature_names,thread_count):"""Initialize Pool from array like data."""if isinstance(data, DataFrame):if feature_names is None:feature_names = list(data.columns)if isinstance(data, Series):data = data.values.tolist()if isinstance(data, FeaturesData):samples_count = data.get_object_count()features_count = data.get_feature_count()else:if len(np.shape(data)) == 1:data = np.expand_dims(data, 1)samples_count, features_count = np.shape(data)pairs_len = 0if label is not None:self._check_label_type(label)self._check_label_empty(label)label = self._label_if_pandas_to_numpy(label)if len(np.shape(label)) == 1:label = np.expand_dims(label, 1)self._check_label_shape(label, samples_count)if feature_names is not None:self._check_feature_names(feature_names,features_count)if cat_features is not None:cat_features = _get_features_indices(cat_features,feature_names)self._check_string_feature_type(cat_features,'cat_features')self._check_string_feature_value(cat_features,features_count, 'cat_features')if text_features is not None:text_features = _get_features_indices(text_features,feature_names)self._check_string_feature_type(text_features,'text_features')self._check_string_feature_value(text_features,features_count, 'text_features')if embedding_features is not None:embedding_features = _get_features_indices(embedding_features, feature_names)self._check_string_feature_type(embedding_features,'embedding_features')self._check_string_feature_value(embedding_features,features_count, 'embedding_features')if pairs is not None:self._check_pairs_type(pairs)if isinstance(pairs, DataFrame):pairs = pairs.valuesself._check_pairs_value(pairs)pairs_len = np.shape(pairs)[0]if weight is not None:self._check_weight_type(weight)weight = self._if_pandas_to_numpy(weight)self._check_weight_shape(weight, samples_count)if group_id is not None:self._check_group_id_type(group_id)group_id = self._if_pandas_to_numpy(group_id)self._check_group_id_shape(group_id, samples_count)if group_weight is not None:self._check_group_weight_type(group_weight)group_weight = self._if_pandas_to_numpy(group_weight)self._check_group_weight_shape(group_weight,samples_count)if subgroup_id is not None:self._check_subgroup_id_type(subgroup_id)subgroup_id = self._if_pandas_to_numpy(subgroup_id)self._check_subgroup_id_shape(subgroup_id,samples_count)if pairs_weight is not None:self._check_weight_type(pairs_weight)pairs_weight = self._if_pandas_to_numpy(pairs_weight)self._check_weight_shape(pairs_weight, pairs_len)if baseline is not None:self._check_baseline_type(baseline)baseline = self._if_pandas_to_numpy(baseline)baseline = np.reshape(baseline, (samples_count, -1))self._check_baseline_shape(baseline, samples_count)self._init_pool(data, label, cat_features, text_features,embedding_features, pairs, weight, group_id, group_weight,subgroup_id, pairs_weight, baseline, feature_names,thread_count)