劃分資料集

如果要在乙個二維資料散點圖中間畫一條線將資料集分開，是按照x劃分還是y劃分呢

from math import log

defcreatedataset
(): dataset = [[1, 1, 'yes'],
[1, 1, 'yes'],
[1, 0, 'no'],
[0, 1, 'no'],
[0, 1, 'no']]
labels = ['no su***cing','flippers']
return dataset, labels
defcalcshannonent
(dataset):
numentries = len(dataset)
labelcounts = {}
for featvec in dataset:
currentlabel = featvec[-1]
if currentlabel not
in labelcounts.keys(): labelcounts[currentlabel] = 0
labelcounts[currentlabel] += 1
shannonent = 0.0
for key in labelcounts:
prob = float(labelcounts[key])/numentries
shannonent -= prob * log(prob,2) 
return shannonent
defsplitdataset
(dataset, axis, value):
retdataset = 
for featvec in dataset:
if featvec[axis] == value:
reducedfeatvec = featvec[:axis]#去除特徵值
reducedfeatvec.extend(featvec[axis+1:])
return retdataset

reducedfeatvec.extend(featvec[axis+1:])# [m:]從m開始全部

>>> import trees

>>> from imp import reload
>>> reload(trees)
'trees' from 'e:\\python\\trees.py'>
>>> mydat,labels=trees.createdataset()
>>> mydat
[[1, 1, 'yes'], [1, 1, 'yes'], [1, 0, 'no'], [0, 1, 'no'], [0, 1, 'no']]
>>> trees.splitdataset(mydat,0,1)
[[1, 'yes'], [1, 'yes'], [0, 'no']]
>>> trees.splitdataset(mydat,0,0)
[[1, 'no'], [1, 'no']]
>>>

def

choosebestfeaturetosplit
(dataset):
numfeatures = len(dataset[0]) -1
baseentropy = calcshannonent(dataset)
bestinfogain = 0.0;bestfeature = -1
for i in range(numfeatures):
featlist = [example[i] for example in dataset]
uniquevals = set(featlist)
newentropy = 0.0
for value in uniquevals:
subdataset = splitdataset(dataset,i,value)
prob = len(subdataset)/float(len(dataset))
newentropy += prob * calcshannonent(subdataset)
infogain = baseentropy - newentropy
if (infogain > bestinfogain):
bestinfogain = infogain
bestfeature = i
return bestfeature

>>> trees.splitdataset(mydat,0,1)

[[1, 'yes'], [1, 'yes'], [0, 'no']]
>>> trees.splitdataset(mydat,0,0)
[[1, 'no'], [1, 'no']]

劃分資料集

import glob import shutil total file total file.txt train train.txt val val.txt bin path data l v bin 總的點雲資料集檔案路徑 如果是自己的路徑，就是 自己點雲資料夾 bin bin train di...

sklearn劃分資料集

train test split是用得最多的資料集劃分包，它的引數有五個 arrays 要切分的資料集，通過傳入兩個，x資料集和目標y test size 測試集樣本大小 random state 隨機種子數 shuffle 是否要對資料集隨機打亂 stratify 可以理解為分層抽樣的設定值，通過...

資料集按類劃分 資料集劃分方法

留出法 直接將資料集d劃分為兩個互斥的集合，乙個為訓練集s，乙個為測試集t，即d s t,s t 在s上進行模型學習，然後用t來評估其測試誤差，作為對泛化誤差的估計。單次使用留出法得到的估計結果往往不夠穩定可靠，在使用留出法時，一般要採用若干次隨機劃分 重複進行模型評估後取平均值作為留出法的評估結果...