Activation Functions

Activation functions introduce non-linearity into neural networks.

Syntax

AI

1
2
3
4
5
6
7
8

# ReLU
f(x) = max(0, x)

# Sigmoid
f(x) = 1 / (1 + e^(-x))

# Tanh
f(x) = (e^x - e^(-x)) / (e^x + e^(-x))

Activation Functions

PYTHON

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import math

def relu(x):
    return max(0, x)

def sigmoid(x):
    return 1 / (1 + math.exp(-x))

def tanh(x):
    return math.tanh(x)

def softmax(values):
    exp_vals = [math.exp(v) for v in values]
    total = sum(exp_vals)
    return [ev / total for ev in exp_vals]

# Test
x = -2
print(f"ReLU({x}) = {relu(x)}")
print(f"Sigmoid({x}) = {sigmoid(x):.4f}")
print(f"Tanh({x}) = {tanh(x):.4f}")
print(f"Softmax([1,2,3]) = {[f\"{s:.3f}\" for s in softmax([1,2,3])]}")

Try It Yourself

Practice

1

Exercise

Practice this concept.

Answer

Write the code as shown above.

Quick Quiz

1

What did you learn?

Concept A

Concept B

Concept C

Concept D

This covers the basics.

Retry Quiz

Interview Questions

Explain this concept.

It is a fundamental AI feature.

Previous

Training a Model

Tutorial Complete

Back to AI