How to stop being afraid of Proguard and start living

Hello, I'm an Android developer and I'm no longer afraid of ProGuard ...

Usually, this utility is remembered when faced with dalvik dex-limit issue or with the requirement to improve the security of the application. Unfortunately, setting up Proguard correctly is far from the first time. I often watched how many, having broken a project, turned off Proguard and turned on Mulditex support, and each time they were a little sad about this, because Proguard helps both to reduce the size of the application and improve its performance.

As a result, I decided to write an article in which I can put all the useful information that I learned during several years of working with Proguard and which could help both the very beginners and those who already know something.

What is it about

Proguard is an open-source utility for optimizing and obfuscating Java code. This tool processes already compiled Java code, so it should work with any JVM language. More precisely, the language itself is indifferent to Proguard, only baytcode is important. All proguard bytecode manipulations can be divided into 3 main categories: Code shrinking , Optimization and Obfuscation .

Code shrinking

Yes, a rather strange idea to write code, and then delete it, but this is the reality of Android development. This, of course, is not about handwritten code (although it happens), but about tons of dead weight, which is brought by libraries of all sorts. Guava , Apache Commons , Google Play Services and other guys can inflate the size of the apk file from 500kb to a couple of tens of megabytes. Sometimes it goes so far that the program cannot compile due to exceeding the Dalvik methods limit . Proguard will help remove all unused code and reduce the size of the application back to several megabytes.

Optimization

In addition to removing unnecessary code, Proguard can optimize the remaining code. In its arsenal there is control flow analysis, data-flow analysis, partial evaluation, static single assignment, global value numbering, liveness analysis. Proguard can perform peephole optimizations, reduce the number of allocations of variables, simplify tail recursions and much more ( wiki ). In addition to such common operations, Proguard has optimizations that are useful specifically for the Android platform, for example, replacing enum classes with int-s, removing logging instructions.

Obfuscation

Finally, Proguard can turn all your code into an unreadable mess by renaming all classes, methods, and fields into sets of random (in fact, not entirely random) letters. This is a very useful option, since anyone can decompile your apk-file, and not everyone will have enough patience to understand the obfuscated code.

Principle of operation

Proguard works in 3 steps in the sequence described above: code shrinking → optimization → obfuscation . Each step is optional.

Optimization step in case of Android SDK is disabled by default.

For Proguard to work, you need to provide 3 components:

Your compiled code is an archive with the class files of your program and all the libraries you use (jar, aar, apk, war, zip, etc.). Proguard only modifies the already compiled code and has nothing to do with the source code.
Configuration file (s) - file (s) containing all the rules, options and settings with which you want to start processing.
Library jars (aar, apks, ...) are the classes of the platform on which your program runs. In the case of Android, this is android.jar . These archives are needed only for the correct analysis of your code, they will not be modified (this does not make sense, because android.jar is "in the phone", we do not have access to it).

Picture of the article
(Picture from the presentation of Jeb Ware, link at the end of the article)

Using library classes and your config files, Proguard determines all entry points to your program ( seeds ). In other words, it defines those classes and methods that can be called externally and which cannot be touched. Then, starting with the detected seeds , Proguard recursively walks around all your code, ticking the “usable” box with everything that I can reach. The rest of the code will be removed. In the config, you must specify at least one entry point. For a standard java program, this is a function of main . In Android, there is no single point of entry into the program; instead, we have standard components (Activity, Service, etc.) that are created and called by the system. Fortunately, we don’t need to specify anything here, the Android SDK will create the necessary config for us.

Related files

After finding all the input points, Proguard writes them into the seeds.txt file.

All the code that Proguard considered unnecessary is written to the usage.txt file. This is a rather strange name for a file containing a remote code, it would be more correct to call it unusage.txt, but we have what we have, just remember about it.

At the obfuscation step, a mapping.txt file will be created containing the pairs <original class name | method | field> -> <obfuscated class name | method | field>. This file is useful when you need to deobfuse program, for example, read stacktrace. Manually mapping files back is not required, in the Android SDK there are retrace and proguardui utilities that will help. Moreover, if you use Fabric Crashlytics, then their gradle plugin can independently find and load this file into the console, so you don’t have to worry about it.

In the case of Android, these files are usually found in app/build/output/mapping/<product-flavor-name>/ .

Proguard also creates a dump.txt file that contains everything that Proguard put into the final archive. He never came to me, but perhaps he will be useful to someone.

How are things in Android

Android Gradle Plugin can run Proguard on its own. All you need to do is enable this option and specify config files.

 buildTypes { <...> release { minifyEnabled true proguardFiles 'proguard-rules.pro', getDefaultProguardFile('proguard-android.txt') } }

minifyEnabled true - enable Proguard at build stage

proguardFiles - list of config files. Rules from all config files will be added to the general list in the order they appear.

proguard-rules.pro is our config file with project-specific rules

getDefaultProguardFile('proguard-android.txt') is a function that returns a standard config file for Android applications. It lies in AndroidSDK/tools/proguard

In fact, the Android SDK has two proguard-android.txt : proguard-android.txt and proguard-android-optimize.txt . In the first of them there is an option -dontoptimize , which turns off all optimizations. If you want to enable optimization, use the second config.

In addition to these standard Android SDK configurations (aapt), the rule set for resources is automatically generated: aapt checks all xml files (including the manifest) to find all activations, services, views, etc. and generate for them the necessary rules. The generated rules can be found in app/build/intermediates/proguard-rules/<flavor>/aapt_rules.txt . You do not need to specify it yourself, Android Gradle Plugin will add these rules automatically.

Picture of the article
(Picture from the presentation of Jeb Ware, link at the end of the article)

Configs

Setting up Proguard is the most basic part of working with it and at the same time the most difficult. Incorrect config can easily break the compilation of the application and the application itself in runtime. All available configuration options are documented in detail on off. site.

Among all the options, I would single out 3 of the most important groups:

keep rules - All possible entry points to the program. Rules telling Proguard which classes or parts of classes should be kept unchanged or which modifications are allowed for specific classes.
optimization tuning - indicate which optimization is permissible, how many optimization cycles need to be done.
work with warnings, errors and debugging

Keep rules

This is a set of options designed to protect your code from the merciless Proguard. In its most general form, this rule looks like this:

 -keep [,modifier,...] class_specification

keep is the most common of these options (there are others), telling Proguard to keep the class itself and all its members (class members): fields and methods.

class_specification - a template pointing to class (s) or parts thereof (class members). The general view of the template is very large, it can be viewed off. documentation . You can refer to it, but in general, you can just remember that we have the opportunity to make a template from the following components:

select all classes with a specific name, package
select all classes that inherit / implement certain classes / interfaces
select all classes with specific modifiers and / or specific annotations
select all methods with a specific name, modifiers, arguments and return value
select all fields with a specific name, modifiers, a specific type.
it is possible to use wildcards

Once again, this is not a strict description of the template, it is rather a list of the features that we have. Here are some examples:

-keep public class com.example.MyActivity
save class com.example.MyActivity

-keep public class * extends android.app.Activity
save all public classes that inherit android.app.Activity

 -keep public class * extends android.view.View { public <init>(android.content.Context); public <init>(android.content.Context, android.util.AttributeSet); public <init>(android.content.Context, android.util.AttributeSet, int); public void set*(...); }

find all public classes that inherit android.view.View and save 3 constructors with certain parameters + all public methods with a void modifier, any arguments and a name starting with set . All other parts of the class can be modified.

-keep class com.habr.** { *; }
save all classes and all their contents in the package com.habr

modifiers - addition to keep-rule:

includedescriptorclasses - in addition to the specified class / method / field, you must save all classes that are found in their descriptors.
includecode - the contents of the method to which this particular rule points, cannot be touched either.
allowshrinking - the classes pointed to by this rule are not input points (seeds) and can be removed, but only if they are not used in the program itself. However, if after the codeshrinking this code remains (due to the fact that someone uses it), this code cannot be optimized / obfuscated.
allowoptimization - the classes indicated by this rule can only be optimized, but not deleted or obfuscated.
allowobfuscation - the classes indicated by this rule can only be obfuscated, but not deleted or optimized.

In addition to keep , there are several options:

-keepclassmembers - indicates that it is necessary to save class members if the class itself is preserved after code shrinking.

-keepclasseswithmembers - indicates that you need to save classes, the contents of which fall under the specified template. For example, -keepclasseswithmembers class * { public <init>(android.content.Context); } -keepclasseswithmembers class * { public <init>(android.content.Context); } - saves all classes that have a public constructor with one argument of type Context .

-keepnames - short for -keep,allowshrinking .

-keepclassmembernames - short for -keepclassmembers,allowshrinking .

keepclasseswithmembernames - short for -keepclasseswithmembers,allowshrinking .

Optimization tuning

The most important option here is the -dontoptimize flag. If present, no optimization will be performed and all other optimization options will be ignored.

There are many optimization options, but the following are the most useful to me:

-optimizations optimization_filter - lists all the methods you want to use. It is better to use the one specified in proguard-android-optimize.txt or its subset. A list of all optimizations can be found here .

-optimizationpasses n is the number of optimization cycles. Several cycles can improve the result. At the same time, Proguard is smart enough to stop the cycles if it sees that the result has not improved since last time.

-assumenosideeffects class_specification - indicates that this method has no side effects and only returns some value. Proguard will remove calls to this method if it finds that the result it returns is not used. Perhaps the most common use of this option is to remove all debug logs: -assumenosideeffects class android.util.Log { public static int d(...); } -assumenosideeffects class android.util.Log { public static int d(...); }

-allowaccessmodification - show all that is hidden :) A great option that allows you to get rid of a heap of artificial accessor-methods for nested classes. Only works in conjunction with -repackageclasses

-repackageclasses - allows all classes to be moved to one specified package. This applies more to obfuscation, but at the same time, gives good results in optimization.

Other useful options

-dontwarn and -dontnote

Proguard is very smart and always reports suspicious places during code analysis, sometimes it’s notes, sometimes it’s a warning. If your build is not going to when Proguard is enabled, be sure to read all the logs it produces, it will write what went wrong and, most likely, even tell you how to fix it. After reading all the messages, you either correct the problem, or ignore the message of one of these options, if you are sure that there is no problem.

For example, it happens that some java-library uses platform-classes that are not in android.jar and Proguard will warn about it. If you are sure that this library is working properly in the Android environment, you can disable this warning -dontwarn java.lang.management.**

-whyareyoukeeping class_specification is a useful option that prints the reason why Proguard decided not to touch this class / method.

-verbose - print more detailed logs and exceptions

-printconfiguration — print the full list of options from all config files that were used, including rules from libraries and generated via aapt.

-keepattributes SourceFile, LineNumberTable - saves meta-information (file names, line numbering) to be able to debug the code in the IDE and get meaningful stacktrace. Be sure to add this option.

Practice

It usually happens like this: turn on Proguard and it breaks the whole project to you giving a ton of errors. Many in this step turn off Proguard and try not to return to it. I will try to give some tips to make this transition process easier.

Decide on starting entry points

If you are an Android developer, everything is extremely simple - just select one of the two standard configs from the Android SDK: proguard-android.txt or proguard-android-optimize.txt , they will take care of everything that should remain untouched.

Check all libraries

Recently, more and more libraries are being distributed with ready-made proguard-configs. Proguard can look inside the archive, find the library config and add it to the other options. Check each library that you use for the presence of such a config.

(the contents of the aar file of one of the libraries)

If you use Google Play Services, the com.google.gms.google-services plugin will select the config you need yourself.

If the authors of the library do not pack the config into the archive, perhaps they took care and wrote the rules on their website, the repository page or in the README file. Try to find the config for the version of the library you are using.

If you can’t find ready-made rules anywhere, you will have to read the logs and solve the problem individually. Most likely, you need to add keep rules for the library code that is broken. Or ignore errors if they do not interfere with the program.

Inspect your code

You can see which code can be sent under the knife, but you should carefully look at all the places where reflection is used one way or another:

Class.forName (...) (the documentation promises that Proguard can identify such code, however, there have been cases, it is worth checking)
Modelki / entity-classes, which are used in seselizatsii, mapping. All classes, the names of the fields (sometimes the classes themselves) which are important to keep (Gson, RealmIO, etc.)
calls to native libraries via JNI

Tests

If a class / method is used only in tests and nowhere else, Proguard will remove this code. This is a common situation if you have TDD :) For this case, I have a separate config where I add classes that are not yet integrated into the project, are not used anywhere, but which need to be tested.

In Android Gradle Plugin, in addition to the proguardFiles instructions, there is also testProguardFiles . This instruction is needed to specify the configs that will be applied to the test application that is generated to test your application when you run instrumentation tests. This is usually used to achieve the same optimization / obfuscation in both apk-files so that there is no desynchronization between them. Link

APK Analyzer

Android Studio has such a great tool. You can open it either through Find Action -> Analyze APK, or by opening the apk file itself in Android Studio. Analyzer shows a lot of useful information, but now we are interested in the code. To see what was eventually packed into the APK file, you need to select the classes.dex file classes.dex

By default, you will be shown exactly the result code that has passed the shrinking and optimisation steps. However, you can click on the Load Proguard mappings ... button, add seeds.txt and usage.txt to see the code that has been removed.

If for some reason Proguard has modified the code you need, find it in the Analyzer and right-click it to select Generate Proguard Keep Rule . Analyzer will generate you a choice of several variants of the rules, from the most general to the most specific, select ONE of them.

For authors of libraries

If you are doing the Android library, you can add a proguard-config to your clients as follows:

 buildTypes { release { consumerProguardFiles 'proguard-rules.pro' } }

In my opinion, it is better not to be zealous with optimizing and obfuscating your library, but to provide this opportunity to your customers. A good tone is to add to the config what clients will still have to add if they include Proguard. However, if you still want to add security, it is obvious that you need to protect the entire pulic API of your library from Proguard, including descriptors and signatures.

R8, DexGuard and Redex

R8 is a new tool from Google in exchange for the current Proguard. Wait, do not try to forget everything that you just read in the article, just consider it as a new Proguard. Google promises to keep the entire public api, so that all configs will work as before. The project is still in beta, but you can try it yourself.

DexGuard is a paid utility from the developers of Proguard. It can be used together or instead of Proguard. It is argued that DexGuard can do everything that Proguard can, but better. Unfortunately, I did not have a chance to try it, if someone has experience, please share.

Redex is another dex optimizer from Facebook. It is reported that with it you can achieve up to 25% increase in productivity and reduce the size of the application by applying the tool to the code already processed by Proguard.

Instead of conclusion

Do not be afraid to use Proguard, do not be lazy and spend some time on the setup. By this you will reduce its size, increase the speed of work, and add the loyalty of your users. At the same time, try to create effective Proguard-configs, do not write "carpet" rules, otherwise an angry Jake Wharton will come to you and scold you.