Fix issue with Partial LSE and casts/instanceof
If PartialLSE encounters an instanceof or check-cast before the
escapes it could not correctly handle it. Due to how java language
code is typically developed and compiled this is generally not a
problem but could lead to incorrect codegen on release builds or
DCHECK failures on debug builds. This fixes the issues by (1) causing
partial LSE to consider check-cast and instance-ofs to be escaping.
This also updates the instruction simplifier to be much more
aggressive in removing instance-of and check-casts.
Test: ./test.py --host
Bug: 186041085
Change-Id: Ia513c4210a87a0dfa92f10adc530e17ee631d006
diff --git a/compiler/optimizing/escape.cc b/compiler/optimizing/escape.cc
index f3f5b15..617833c 100644
--- a/compiler/optimizing/escape.cc
+++ b/compiler/optimizing/escape.cc
@@ -41,6 +41,13 @@
if (!escape_visitor(user)) {
return;
}
+ } else if (user->IsCheckCast() || user->IsInstanceOf()) {
+ // TODO Currently we'll just be conservative for Partial LSE and avoid
+ // optimizing check-cast things since we'd need to add blocks otherwise.
+ // Normally the simplifier should be able to just get rid of them
+ if (!escape_visitor(user)) {
+ return;
+ }
} else if (user->IsPhi() ||
user->IsSelect() ||
(user->IsInvoke() && user->GetSideEffects().DoesAnyWrite()) ||
@@ -108,6 +115,9 @@
// Ignore already known inherent escapes and escapes client supplied
// analysis knows is safe. Continue on.
return true;
+ } else if (escape->IsInstanceOf() || escape->IsCheckCast()) {
+ // Ignore since these are not relevant for regular LSE.
+ return true;
} else if (escape->IsReturn()) {
// value is returned but might still be singleton. Continue on.
*is_singleton_and_not_returned = false;