SageMaker
I wasn’t familiar with the concept of notebooks before this exercise.
I had to learn that the notebook itself is essential a scripting environment.
So when a code field is present, Shift + Enter…does “invisible” or “background” work.
Not too different from the CLI terminal where unless there’s an error you might not get an output.
Still a notebook feels more like a webpage than a terminal.
Think of a notebook like an excel spreadsheet that goes down vertically.
Cells are the “fields” where you enter Python code. Shift+Enter execute those codes. You in SageMaker will either see the output directly below the cell or to the left in terms of adding columns, splitting data, etc
The use case was 310 patients with varying orthopedic states of health.
To start off, I imported data with the following templates:
Data was imported with 4 command cells:
This command cell downloaded, stored and unzipped data:
f_zip = ‘http://fileurl.zip'
r = requests.get(f_zip, stream=True)
fileurl_zip = zipfile.ZipFile(io.BytesIO(r.content))
fileurl_zip.extractall()
This command cell loads the data into the dataframe so it can be used:
data = arff.loadarff(‘column_2C_weka.arff’)
df = pd.DataFrame(data[0])
Weka is Waikato Environment for Knowledge Analysis. This is a software used at the University of Waikato.
.ARFF is the fileformat for WEKA.
This command cell creates directories for classication
class_mapper = {b’Attribute1’:1,b’Attribute2’:0}
df[‘class’]=df[‘class’].replace(class_mapper)
Get Mr. Raph’s stories in your inbox
Join Medium for free to get updates from this writer.
I then ran these cell commands to split the date up:
from sklearn.model_selection import train_test_split
train, test_and_validate = train_test_split(df, test_size=0.2, random_state=42, stratify=df[‘class’])
Test size of 0.2 stands for 20 percent to not be trained and used for data sourcing. The other 80 percent will train the large language model.
This command indicates using 50 percent of that previous 20 percent for, which is 10% for data sourcing. The other 50 percent will be used to verify data in the model.
test, validate = train_test_split(test_and_validate, test_size=0.5, random_state=42, stratify=test_and_validate[‘class’])
So overall, 80% will be used to train, 10% will be used for data sourcing and 10% will be used for verification.
I then a command cell to examine the 3 differing types of orthopedic states:
print(train.shape)
print(test.shape)
print(validate.shape)
I then assessed how data would be ordered:
print(train[‘class’].value_counts())
print(test[‘class’].value_counts())
print(validate[‘class’].value_counts())
I transferred the data to csvs and uploaded to an S3 bucket:
I imported the XGBoost Uniform Resource Identifier:
import boto3
from sagemaker.core.image_uris import retrieve
container = retrieve(framework=’xgboost’, region=boto3.Session().region_name, version=’1.0–1')
I ran a hyperparameter to determine how the training would be executed:
hyperparams={“num_round”:”42",
“eval_metric”: “auc”,
“objective”: “binary:logistic”}
An estimator function is used to configure settings such as how many instances and what type of instance. The estimator uses channels to allow data into the model, so a training and validation channels were made.
I also learned that “fit” is a synonym for training models. I attempted to view the .csv files in the bucket, but didn’t have sufficient IAM permissions:





