Many social media professionals have probably been in this situation. A post is ready. The text has been chiselled to perfection. The best possible picture has been painstakingly selected. Finally, the post goes live… and it just does not look that great. Often, this problem is beyond control of the person responsible for publishing content.

Twitter has been aware of these types of glitches for a while, especially when tweeting photos. Images come in many shapes and sizes, so Twitter has been automatically cropping them in a way that delivers a consistent UI experience. A company representative explains the most common issue with this approach:

Previously, we used face detection to focus the view on the most prominent face we could find. While this is not an unreasonable heuristic, the approach has obvious limitations since not all images contain faces. Additionally, our face detector often missed faces and sometimes mistakenly detected faces when there were none. If no faces were found, we would focus the view on the center of the image. This could lead to awkwardly cropped preview images.

Twitter claims this will no longer be a problem. The microblogging platform transformed its approach to making images look better. A lot of work went into a matter that might seem trivial at first glance — but better looking pictures simply matter to users.

First, the company closely examined researchers’ conclusion that it is best to focus on “salient image regions”. These are the parts of pictures people usually pay the most attention to. Salient image regions have been identified by researchers using eye trackers to capture the pixels on which individuals tend to focus on.

How saliency prediction works

Then, neural networks are used to predict the saliency of new images. But, as Twitter notes, these sophisticated networks are simply too slow to process all of the visual content being uploaded in real time. Accordingly, a knowledge distillation technique is used to train a smaller network to mimic the more powerful one. Next, the computationally intensive but ineffective parts of the neural network are identified and eliminated.

By combining these two techniques, Twitter can crop images with better end results — and ten times faster than via previous processes. Remarkably, the company is claiming that these are the results before any optimizations were introduced. These are impressive! And better still, all Twitter users will be able to enjoy them shortly. An update is being rolled out globally and should soon be available to everyone. The effects of all this complicated work are clearly visible in the images shared by Twitter.

For anyone interested in the topic and the approach taken by Twitter, more details are available on the company’s blog.

Images before new algorithm has been used


Images after new algorithm has been used

Would you like to receive the latest news from Okimo Clinic? Subscribe to our monthly newsletter by clicking here.