8 Compression Techniques for Machine Learning Developers

Novel network quantization methods: SLQ and MLQ.

[{"selector":"#anim-78cf2a56-eb5b-4e85-b5c6-dd51f3a72e53","keyframes":{"transform":["translate3d(-115.14196%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b51c32c2-5981-4b7d-962a-fed488abbab5","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-3d390ffd-6b79-404d-8306-e1421c8d981f","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Efficient whole-network rank configuration proposal.

[{"selector":"#anim-86b4eaf5-8277-4242-a491-c852a98d4716","keyframes":{"transform":["translate3d(-114.24333%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-97c9b4a1-c637-4057-83d5-f23b3f374fa8","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-79d2229b-cab9-44e3-a744-eca4dfac73f9","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Google's 3LC: Lightweight, balanced traffic compression.

[{"selector":"#anim-faad1ab0-1f2a-469f-9187-397fb13e1f09","keyframes":{"transform":["translate3d(-114.8265%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-338e944c-b306-4be7-863f-211d02258926","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ec3863df-b397-4910-9d7b-711fd1851776","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Universal DNN compression using randomization.

[{"selector":"#anim-5fb024a7-895b-400a-9d46-03e3bc024d9a","keyframes":{"transform":["translate3d(-115.14196%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-61dbc5f0-37fb-4318-8743-0733c7968f0c","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-825e5ef0-da39-415d-947f-8c172bfa5aac","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Transform coding and clustering for efficient encoding.

[{"selector":"#anim-445c18e8-7dc6-4ef0-9e6c-819838d6ce27","keyframes":{"transform":["translate3d(-114.8265%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-972e96a4-bd69-4cfc-838d-99ff8131c102","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4c35e716-1b3e-4464-94f0-40a539dafe45","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Weightless encoding achieves 496x weight compression.

[{"selector":"#anim-ec87d267-d387-4d70-a9e7-0de1bd1f0326","keyframes":{"transform":["translate3d(-115.14196%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-9cf4cb7f-69d8-4328-8d8f-c4e2bc7df4b3","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-37c4c6a5-f2b7-4066-8ae9-09d4e1e1e478","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Adaptive estimators enhance compression sensitivity.

[{"selector":"#anim-3180cda5-390f-4dc9-b3ca-637c2065c3b1","keyframes":{"transform":["translate3d(-115.2381%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-fb70c373-9201-48e2-a9b3-65d01736d7b1","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-be224470-c265-420e-b877-7a88920fc363","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

MLPrune automates compression ratio decisions for layers.

[{"selector":"#anim-683243b4-82ae-4ff1-ba29-848b9b7ce883","keyframes":{"transform":["translate3d(-115.2381%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-f6342cf8-c186-42f9-9a44-fc3dc79da26e","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-bbb1bda7-f6b6-4a1b-af11-2191adb3ebd1","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]