In the deluge of WWDC announcements, there was the brief mention of a technology that is going to save you a ton of space on your devices: HEVC and HEIF. These are two file formats, one for video and one for photos, respectively.
Apple is already building these into hardware and software on iOS and the Mac, with support available in the upcoming iOS 11 and macOS High Sierra updates.
HEVC stands for High Efficiency Video Coding, while HEIF stands for High Efficiency Image file Format.
Codecs and Compression 101
HEVC is industry standard, not just a new Apple only file format. HEIF is an image format based on the same principals as HEVC. Each is designed to save you space on your devices, as well as bandwidth when streaming video.
Because of the proliferation of HD video, as well as short-form videos to social networks, Apple says now is the time to push the new codecs and move the whole industry forwards.
Simply put, codecs are used to shrink media and image files down to save space on the disk. This is a generalization, for in-depth details check out our explainer on codecs for a precise definition. The file formats used to save your media that can implement various kinds of codecs are named containers.
HEVC is a derivative of the popular H.264 format, and similarly handles compression. That is why the ITU-T standards body gives it the name of H.265. H.264 is the most pervasive format for online video, so it is likely that you are using it most days. HEIF uses the same formatting as HEVC and applies it to image compression.
What Makes HEVC Different?
Forcing users and developers alike to adopt any new format can be painful. What are we gaining by Apple pushing this is the default format? The highlights from the WWDC talk promise a 40% improvement in compression. For video captured from the iOS camera, files can be half the size of those encoded with H.264.
These improvements come from the small tweaks made to the way that video is encoded. Like H.264, HEVC handles frames in a series of “macroblocks,” but HEVC allows for much larger block sizes. These larger block sizes allow the greatest compression improvement.
There’s improved motion coding as well, including support for measuring motion at a sub-pixel level. That change should improve artifacts when a video is in motion, while the new codec should improve video quality overall.
What About HEIF?
The most straightforward way to describe the HEIF format is applying the principals of HEVC to images. HEIF is another industry standard, not just an Apple-only format. Like HEVC, HEIF has an improved compression rate, Apple reporting almost twice an improvement.
Definitely a time saver for uploads: video we just encoded is 1.74GB h.264, 795MB HEVC. Remember that CPU encoding is a lot slower though.
— Digital Foundry (@digitalfoundry) June 23, 2017
HEIF has better support for animated photos, so it is a natural fit for Live Photos and animated GIFs. Photographers should like the format’s improved support for non-destructive edits. That means as you make changes to the original file, the original is always available. The edit data saves as a transformation function using the HEVC format.
Compatible with the ISO file format, the supported extension in the Apple ecosystem is .HEIC. There are different formats for other codes introduced at WWDC: .AVCI for H.264 encoding and .HEIF for all other codecs.
Are There Hardware Constraints?
Apple is going all-in on HEVC and HEIF. The new formats will be available in iOS 11 and macOS High Sierra. However, there is already hardware support built into existing iOS devices.
Anything with an A9 processor or later is already able to use hardware decoding. The A10 processor has hardware encoding as well (allowing you to write to the format). Hardware encoding means a lower hardware overhead for these devices. On the Mac side, Intel 6th-generation processors or later have hardware decoding. Hardware encoding is limited to Intel’s 7th-generation processors.
All other supported systems use software encoding and decoding. The higher resources for software decoding means some devices will not play videos. In real terms, that means iPhone 6s (and later) owners can use hardware decoding, while iPhone 7 and later will be able to use hardware encoding.
When Can I Use It?
HEVC and HEIF are built into the APIs and frameworks for iOS and macOS. Many developers are going or have a painless transition to supporting these formats.
Even if you do not watch that much video, Photos is going all in on HEIF. That means you are going to start using it almost immediately when your iPhone upgrades. It is not yet clear if that means that Apple will update your existing library to the new format. That choice could give you some returned space on your device.
Then I realized that only my PC & iPhone can actually play HEVC videos because nothing else is powerful enough. The future isn't here yet.
— Chief Oddball (@Chief_Oddball) June 12, 2017
Apple is also looking to reduce the transcoding your device needs to do. They are going to check the destination of your video for compatibility. For example, sharing videos between apps with the share sheet: if HEVC is supported, the app will get the decoded video to start working with. Devices sharing via AirDrop will check for comparability as well. It is an impressive way to save CPU cycles for older devices.
Under the Hood
Video and image codecs are not going to be the highlight of the keynote for most people. However, they will change the way that your hardware and software work. It shows that Apple focused on improving the fundamentals of the OS. Instead of just chasing Keynote features for the tech press.
Changing an ecosystem’s preferred format for both video and images is not going to simple. You can expect that not all developers will get the memo, so expect to give some developers time to adjust to the changes.
Were you desperate for Apple to make the change to H.265? Let us know in the comments.