Sowing Success: How Machine Learning Helps Farmers Select the Best Crops
Measuring essential soil metrics such as nitrogen, phosphorous, potassium levels, and pH value is an important aspect of assessing soil condition. However, it can be an expensive and time-consuming process, which can cause farmers to prioritize which metrics to measure based on their budget constraints.
Farmers have various options when it comes to deciding which crop to plant each season. Their primary objective is to maximize the yield of their crops, taking into account different factors. One crucial factor that affects crop growth is the condition of the soil in the field, which can be assessed by measuring basic elements such as nitrogen and potassium levels. Each crop has an ideal soil condition that ensures optimal growth and maximum yield.
A farmer reached out to you as a machine learning expert for assistance in selecting the best crop for his field. They've provided you with a dataset called soil_measures.csv, which contains:
"N": Nitrogen content ratio in the soil"P": Phosphorous content ratio in the soil"K": Potassium content ratio in the soil"pH"value of the soil"crop": categorical values that contain various crops (target variable).
Each row in this dataset represents various measures of the soil in a particular field. Based on these measurements, the crop specified in the "crop" column is the optimal choice for that field.
In this project, you will apply machine learning to build a multi-class classification model to predict the type of "crop", while using techniques to avoid multicollinearity, which is a concept where two or more features are highly correlated.
# All required libraries are imported here for you.
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
import seaborn as sns
from sklearn.metrics import f1_score
## Task 1
# Load the dataset
crops = pd.read_csv("soil_measures.csv")
# crops.head()
# crops.describe()
# crops.dtypes
# Check the number of crops
num_crops = crops['crop'].nunique()
print(f"Number of crops: {num_crops}")
# Check for missing values
missing_values = crops.isnull().sum()
print("Missing values:")
print(missing_values)
# Verify that data in each potential feature column is numeric
numeric_cols = crops.select_dtypes(include=['float64', 'int64']).columns
non_numeric_cols = crops.select_dtypes(exclude=['float64', 'int64']).columns
print("Numeric columns:")
print(numeric_cols)
print("Non-numeric columns:")
print(non_numeric_cols)
## Task 2
# Split the data into training and test sets
X = crops.drop('crop', axis=1) # Features (all columns except 'crop')
y = crops['crop'] # Target variable ('crop' column)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Print the shape of the training and test sets
print("Training set shape:", X_train.shape)
print("Test set shape:", X_test.shape)
## Task 3
# # Loop over each feature and fit a Logistic Regression model
# for feature in X_train.columns:
# # Select the current feature
# X_train_feature = X_train[[feature]]
# X_test_feature = X_test[[feature]]
# # Create and fit a Logistic Regression model
# model = LogisticRegression(max_iter=2000, multi_class='auto')
# model.fit(X_train_feature, y_train)
# # Predict the crop type
# y_pred = model.predict(X_test_feature)
# # Calculate the f1_score
# f1 = f1_score(y_test, y_pred, average='weighted')
# print(f"Feature: {feature}, F1 Score: {f1}")
## Task 4
# # Perform correlation analysis
# correlation_matrix = crops.drop('crop', axis=1).corr()
# # Find highly correlated feature pairs
# highly_correlated_pairs = []
# threshold = 0.8 # Set the correlation threshold
# for i in range(len(correlation_matrix.columns)):
# for j in range(i+1, len(correlation_matrix.columns)):
# if abs(correlation_matrix.iloc[i, j]) > threshold:
# pair = (correlation_matrix.columns[i], correlation_matrix.columns[j])
# highly_correlated_pairs.append(pair)
# print("Highly correlated feature pairs:")
# for pair in highly_correlated_pairs:
# print(pair)
# # # Split the data into training and test sets
# # X = crops.drop('crop', axis=1) # Features (all columns except 'crop')
# # y = crops['crop'] # Target variable ('crop' column)
# # X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# # Loop over each feature and fit a Logistic Regression model
# for feature in X_train.columns:
# if feature not in [pair[0] for pair in highly_correlated_pairs] and feature not in [pair[1] for pair in highly_correlated_pairs]:
# # Select the current feature
# X_train_feature = X_train[[feature]]
# X_test_feature = X_test[[feature]]
# # Create and fit a Logistic Regression model
# model = LogisticRegression(max_iter=2000, multi_class='auto')
# model.fit(X_train_feature, y_train)
# # Predict the crop type
# y_pred = model.predict(X_test_feature)
# # Calculate the f1_score
# f1 = f1_score(y_test, y_pred, average='weighted')
# print(f"Feature: {feature}, F1 Score: {f1}")
## Task 5
# Perform correlation analysis
correlation_matrix = crops.drop('crop', axis=1).corr()
# Find highly correlated feature pairs
highly_correlated_pairs = []
threshold = 0.8 # Set the correlation threshold
for i in range(len(correlation_matrix.columns)):
for j in range(i+1, len(correlation_matrix.columns)):
if abs(correlation_matrix.iloc[i, j]) > threshold:
pair = (correlation_matrix.columns[i], correlation_matrix.columns[j])
highly_correlated_pairs.append(pair)
# Create the final feature list
final_features = [feature for feature in crops.columns if feature not in [pair[1] for pair in highly_correlated_pairs]]
# # Split the data into training and test sets
# X = crops[final_features] # Features (final selected features)
# y = crops['crop'] # Target variable ('crop' column)
# X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Train a Logistic Regression model
log_reg = LogisticRegression(max_iter=2000, multi_class='auto')
log_reg.fit(X_train, y_train)
# Predict the crop type
y_pred = log_reg.predict(X_test)
# Evaluate the model performance using f1_score
model_performance = f1_score(y_test, y_pred, average='weighted')
print("Model Performance (F1 Score):", model_performance)