Embedding APIs and databases work best in batches, say 100 texts per request. Write a generator that groups any iterable into batches lazily, so it works on huge files and even infinite streams.
Implement batched(iterable, n):
n items. The last tuple may be shorter. Never yield an empty tuple.iterable only as needed. Don't call list() on the input.ValueError if n < 1.itertools.batched. Write it yourself.Example
list(batched("abcdefg", 3)) # [('a', 'b', 'c'), ('d', 'e', 'f'), ('g',)]