I used prompt engineering to get a translation quality and readability better than what I can get …

Progressive summarization works pretty well - I'm using that for https://findsight.ai You can even get lesser LLMs to do the bulk reduction that have GPT clean it up on the way to even less content. Admittedly, that does take a lot of prompt engineering, chunk selection and reinforcement though (LLM supervising LLM).

If telling Chat GPT that it has “decades of experience” leads to better results, why not say centuries of experience instead? If you don’t know how every token of input affects the output, is engineering an accurate way to describe what you’re doing?

Very interesting use of ChatGPT and prompt engineering. For your problem, summarizing a large document, splitting the document into smaller parts is indeed the way to go. I also had problems myself with operating on large documents. In my case, I had an insurance policy that I wanted to extract information from. My solution: use the OpenAI API to convert the document to OpenAI's embeddings and saving those embeddings to a vector database. Then, use similarity search on the database to find chunks of the document that might be related to my query and pass only those chunks to GPT for the information extraction prompt. I plan to create a guide on how to tackle these problems after I consolidate my findings.

I've actually coded something like this. The trick is to split it in a smart way. What I did is the following: 1. Download the video 2. Split it in max length for whisper 3. Used whisper for transcription on the chunks 4. Used gpt for cleaning the transcription 5. Unioned the cleaned transcribed chunks 5. Used the langchain "refine" function of the summary chain It is quite slow and expensive because the refine summary is doing a lot of calls, but the result is amazing. And you don't need a vector DB for that because the summary is serial. For Q&A tho, an embeddings DB is a must

Try Universal Summarizer. No token limit. https://kagi.com/summarizer