Live and let's learn

This page contains two complete problem sets designed for early-stage machine learning study: a Math Foundations Set and a Python/NumPy Programming Set. These problems align with the first two weeks of Andrew Ng’s Machine Learning specialization and are intended for structured notebook practice.

---

🧮 Problem Set A — Math Foundations

1. Dot Product & Linear Model

Given:

• x = [2, 3, −1]ᵀ
• w = [0.5, −2, 4]ᵀ
• b = 1

Tasks:

1. Compute z = wᵀx + b
2. Compute g(z) = 1 / (1 + e^(−z))
3. Interpret the resulting probability.

---

2. Sigmoid → Logistic Model Reasoning

Show the logical chain:

• g(z) = 1 / (1 + e^(−z))
• z = wᵀx + b
• ŷ = g(wᵀx + b)

Explain why squared error is not appropriate for logistic regression.

---

3. Bernoulli Likelihood → Logistic Cost

For a single example with label y ∈ {0, 1}:

1. Write the likelihood: P(y|x; w) = ŷ^y · (1 − ŷ)^(1−y)
2. Take the log.
3. Take the negative.
4. Show the final logistic cost function.

---

4. Gradient of Logistic Regression

Derive ∂J/∂wᵢ for:

• J = −[y·log(ŷ) + (1 − y)·log(1 − ŷ)]
• Use: d/dz g(z) = g(z)·(1 − g(z))

---

5. Linear Regression Cost Gradient

Given:

• ŷ = wᵀx + b
• J = ½·(ŷ − y)²

Derive:

• ∂J/∂wᵢ
• ∂J/∂b

---

6. Decision Boundary Interpretation

Show that the logistic regression decision boundary is:

• wᵀx + b = 0

Explain why this represents a hyperplane.

---

7. Polynomial Feature Expansion

Given x = (x₁, x₂), construct:

• φ(x) = (1, x₁, x₂, x₁², x₁x₂, x₂²)

Explain how this changes the decision boundary shape.

---

🐍 Problem Set B — Python/NumPy Practice

1. Implement the Sigmoid Function

def sigmoid(z):
    return 1 / (1 + np.exp(-z))


Test it on:

z = np.array([-2, 0, 2])


---

2. Compute Linear Model Output

Given:

w = np.array([0.5, -2, 4])
x = np.array([2, 3, -1])
b = 1


Compute:

z = w @ x + b
y_hat = sigmoid(z)


---

3. Vectorized Linear Regression Prediction

def predict(X, w, b):
    return X @ w + b


Test on:

X = np.array([[1,2],[3,4],[5,6]])
w = np.array([0.1, 0.2])
b = 0.5


---

4. Linear Regression Cost Function

def cost(X, y, w, b):
    m = len(y)
    y_hat = X @ w + b
    return (1/(2*m)) * np.sum((y_hat - y)**2)


---

5. Gradient Descent Step

def gradient_step(X, y, w, b, alpha):
    m = len(y)
    y_hat = X @ w + b
    dw = (1/m) * (X.T @ (y_hat - y))
    db = (1/m) * np.sum(y_hat - y)
    w = w - alpha * dw
    b = b - alpha * db
    return w, b


---

6. Logistic Regression Prediction

def logistic_predict(X, w, b):
    return sigmoid(X @ w + b)


---

7. Logistic Cost Function

def logistic_cost(X, y, w, b):
    m = len(y)
    y_hat = sigmoid(X @ w + b)
    return -(1/m) * np.sum(y*np.log(y_hat) + (1-y)*np.log(1-y_hat))


---

8. Logistic Gradient Step

def logistic_gradient_step(X, y, w, b, alpha):
    m = len(y)
    y_hat = sigmoid(X @ w + b)
    dw = (1/m) * (X.T @ (y_hat - y))
    db = (1/m) * np.sum(y_hat - y)
    w = w - alpha * dw
    b = b - alpha * db
    return w, b


When I lok