Counting MLP Parameters and Memory

easy
Unit 7 · NLP0/3 to milestone

Implement:

  • mlp_params(sizes, bias=True): parameter count of an MLP with layer widths sizes, e.g. [784, 256, 10] has two linear layers
  • training_memory_bytes(n_params, bytes_per_param=4, optimizer="adam"): memory for weights + gradients + optimizer state, where Adam keeps 2 extra values per parameter, SGD with momentum 1, and plain SGD 0
#architecture#memory2 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Counting MLP Parameters and Memory · TechGradify