PlayStation patent describes AI frame generation to reduce latency

Sony Interactive Entertainment applied for intellectual property rights regarding a system utilizing an artificial intelligence autoencoder to forecast upcoming visuals ahead of rendering. Unlike conventional frame generation that requires two rendered frames to create an intermediate one, this method uses frame extrapolation. The system uses an encoder and decoder neural network to analyze previous frames, motion vectors, and recent user input. The encoder converts a frame into an image embedding, which the decoder then uses to generate a synthetic frame.

This approach may help maintain smoother gameplay by displaying a predicted frame while hardware renders the next real one. The documentation also outlines a simpler alternative method without artificial intelligence that repositions pixel blocks according to their movement. The patent notes that predicting a future frame is difficult and can lead to visual inaccuracies during sudden camera movements, unexpected player actions, or when new objects appear. Should hardware developers integrate this system into future gaming devices, it could reduce delay more effectively than current methods.