Closed Bug 1085593 Opened 10 years ago Closed 10 years ago

[SHB] The "Enter SIM PIN" page flashes when Software Home Button is enabled

Categories

(Core :: Graphics, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

()

RESOLVED FIXED
2.1 S8 (7Nov)
blocking-b2g 2.1+
Tracking Status
b2g-v2.1 --- ?
b2g-v2.2 --- affected
b2g-master --- ?

People

(Reporter: smiko, Assigned: BenWa)

References

()

Details

(Whiteboard: [2.2-Daily-Testing][shb-enabled][POVB])

Attachments

(4 files, 1 obsolete file)

Attached file PIN.txt
Description: The "Enter SIM PIN" page flickers when launched with the software home button enabled.

Repro Steps:
1: Update a Flame to 20141020040204
2: Open Settings > SIM Manager > SIM Security and create a SIM PIN
3: Open Settings > Developer Menu and enable Software Home Button
4: Long press the power button and select Restart
5: Unlock the phone.

Actual:
The "Enter SIM PIN" page flashes black before displaying

Expected:
The page is displayed without any flashing/flickering

Device: Flame 2.2 (319mb/ full flash)
BuildID: 20141020040204
Gaia: dc496d04907dd314f9736ff78bab3bd27156f79a
Gecko: ca6b46cbca3b
Gonk: 05aa7b98d3f891b334031dc710d48d0d6b82ec1d
Version: 36.0a1 (2.2)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:36.0) Gecko/36.0 Firefox/36.0

Notes:
1: This issue ONLY occurs with the software home button enabled

Repro frequency: 100%

See attached: logcat

Video clip: http://youtu.be/4tBoMTlYVWQ
This issue Does occur on Flame 2.2 (319mb/full flash/v188 base)

Actual result: The "Enter SIM PIN" page flashes black before displaying

Device: Flame 2.2
BuildID: 20141020040204
Gaia: dc496d04907dd314f9736ff78bab3bd27156f79a
Gecko: ca6b46cbca3b
Gonk: 05aa7b98d3f891b334031dc710d48d0d6b82ec1d
Version: 36.0a1 (2.2)
Firmware: V188
User Agent: Mozilla/5.0 (Mobile; rv:36.0) Gecko/36.0 Firefox/36.0

____________________________________________________________________________________________________________________________________________

This issue does NOT occur on Flame 2.1 (319mb/full flash/v180) or Flame 2.1 (319mb/full flash/v188)

Actual result: The page is displayed without any flashing/flickering

Device: Flame 2.1
BuildID: 20141020001201
Gaia: 2904ab80816896f569e2d73958427fb82aebaea5
Gecko: 12dc9b782f2a
Gonk: 05aa7b98d3f891b334031dc710d48d0d6b82ec1d
Version: 34.0 (2.1)
Firmware: V180
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

Device: Flame 2.1
BuildID: 20141020001201
Gaia: 2904ab80816896f569e2d73958427fb82aebaea5
Gecko: 12dc9b782f2a
Gonk: 05aa7b98d3f891b334031dc710d48d0d6b82ec1d
Version: 34.0 (2.1)
Firmware: V188
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0
QA Whiteboard: [QAnalyst-Triage?]
Flags: needinfo?(pbylenga)
Whiteboard: [2.2-Daily-Testing]
[Blocking Requested - why for this release]:
Poor user experience with high visibility.

Confirmed with Reporter that he reported 2.1 as unaffected because the screen doesn't recover from the overlap. therefore, I believe this is related to bug 1082071.
blocking-b2g: --- → 2.2?
QA Whiteboard: [QAnalyst-Triage?]
Depends on: 1082071
Flags: needinfo?(pbylenga)
Whiteboard: [2.2-Daily-Testing] → [2.2-Daily-Testing] [shb-enabled]
Yup, this is definitely caused by bug 1082071, which will soon be uplifted to 2.1.

The problem is that when the screen unlocks, we need to trigger a system wide screen resize when SHB is enabled. If we don't trigger this, many screens like the one in bug 1082071 will be the wrong size. App windows are also affected.

Alive, is there anything we can do about the flickering here? Perhaps we could wait to dismiss the lockscreen until after the resize has been propagated across all window managers?
Flags: needinfo?(alive)
Whiteboard: [2.2-Daily-Testing] [shb-enabled] → [2.2-Daily-Testing] [shb-enabled][systemsfe]
(In reply to Michael Henretty [:mhenretty] from comment #3)
> Yup, this is definitely caused by bug 1082071, which will soon be uplifted
> to 2.1.
> 
> The problem is that when the screen unlocks, we need to trigger a system
> wide screen resize when SHB is enabled. If we don't trigger this, many
> screens like the one in bug 1082071 will be the wrong size. App windows are
> also affected.
> 
> Alive, is there anything we can do about the flickering here? Perhaps we
> could wait to dismiss the lockscreen until after the resize has been
> propagated across all window managers?

Offline discussed =)
Flags: needinfo?(alive)
Bad UX.
blocking-b2g: 2.2? → 2.1+
Assignee: nobody → mhenretty
WIP patch. We still see the flashing, but it's much less pronounced. I think we still need to do more here.
Here's a real strange thing I found, this only reproduces if the app behind the SIM PIN dialog is the homescreen, and only if it is scrolled all the way to the top. I wonder if this has something to do with the statusbar being transparent.
Target Milestone: --- → 2.1 S8 (7Nov)
After much investigation and profiling, I'm pretty sure this is a graphics issue. The flashing only seems to occur when the screen over which the SIM PIN dialog is displayed has a transparent statusbar. Furthermore, if you enable the FPS hud, you see very similar artifacting on the screen as bug 1089040 [1]. There's a good chance these issues are related. After systematically disabling js that runs on 'system-resize' using the profiler as a guide, I fairly certain this is something lower.

BenWa, could you help me investigate this issue? The line that causes the flashing/artifacting is when we update the height of the SIM PIN dialog here [2]. But it only happens when closing the lockscreen when the statusbar is transparent. Let me know if I can collect any more information here of use to you.


1.) https://www.youtube.com/watch?v=tmjGR6uyClI
2.) https://github.com/mozilla-b2g/gaia/blob/78be6285b44310cc45fd3c4dd41bb2e71bb7ce69/apps/system/js/system_dialog.js#L177
Flags: needinfo?(bgirard)
Lets put this on Milans radar since it might become a gfx blocker.
Flags: needinfo?(milan)
Thanks for the heads up.  Quick question - which devices don't have the hardware button and need the software button?
(In reply to Milan Sreckovic [:milan] from comment #10)
> Thanks for the heads up.  Quick question - which devices don't have the
> hardware button and need the software button?

Tako will have SHB.
Assignee: mhenretty → nobody
Component: Gaia::System::Lockscreen → Graphics
Product: Firefox OS → Core
Whiteboard: [2.2-Daily-Testing] [shb-enabled][systemsfe] → [2.2-Daily-Testing] [shb-enabled]
This only becomes 2.1+ if we actually uplift bug 1082071 to 2.1 - we should not do that and introduce a regression until we have the regression taken care of.  I've commented in bug 1082071 to that effect.  In the meantime, we can take a look at this on the trunk.  I have serious doubts that this is graphics, but Benoit can help find the problem.
Assignee: nobody → bgirard
Flags: needinfo?(milan)
(In reply to Michael Henretty [:mhenretty] from comment #8)
> After much investigation and profiling, I'm pretty sure this is a graphics
> issue.

Can you list what you tried on the bug and your though process for coming to this conclusion. To me it doesn't feel like a graphics issue but perhaps if you shared your work so far that would help.

Specially I'd like to know that the app isn't just asking for the wrong content on that frame like we're seen many thing in these bugs before jumping to thinking it's a graphics issue. Also when a document is loaded we only block rendering for a fraction of a second so if we have something delay the document load we will show whatever we've loaded up to that point which may very well be a black background. This is a pretty common mistake.

> The flashing only seems to occur when the screen over which the SIM
> PIN dialog is displayed has a transparent statusbar.

Can we get a testcase reduction of that? Trying to simplify the app and system down to a minimal set of feature/complexity will help narrow down the bug. Particularly since the bug is very short lived debugging is more difficult.

> Furthermore, if you
> enable the FPS hud, you see very similar artifacting on the screen as bug
> 1089040 [1]. There's a good chance these issues are related.

If that's the case then yes this is likely a graphics bug. But the original video doesn't show any artifacts, it looks more like bad/slow content provided for that frame. We could also have 2 or more bugs here. The artifact in that video are certainly a graphics issue. I'll look at that issue first (which wont fix this if this isn't a graphics issue).

> After
> systematically disabling js that runs on 'system-resize' using the profiler
> as a guide, I fairly certain this is something lower.

Can you share your profile and what you saw in it that makes you think that so that I can follow your analysis? Can you show me what you disabled?

> 
> BenWa, could you help me investigate this issue? The line that causes the
> flashing/artifacting is when we update the height of the SIM PIN dialog here
> [2]. But it only happens when closing the lockscreen when the statusbar is
> transparent. Let me know if I can collect any more information here of use
> to you.
> 
> 
> 1.) https://www.youtube.com/watch?v=tmjGR6uyClI
> 2.)
> https://github.com/mozilla-b2g/gaia/blob/
> 78be6285b44310cc45fd3c4dd41bb2e71bb7ce69/apps/system/js/system_dialog.js#L177

Can someone confirm if this is a regression? If so we should run a regression window. I'd be particularly curious to know if the offender was bug 1085223 based on the video in [2] if the regression is recent. Even if this didn't work in 2.1 we can still run a regression along the gecko changes that we're made in 2.2 to see if a change broke it.
Flags: needinfo?(bgirard) → needinfo?(mhenretty)
I took a video of the problem https://www.youtube.com/watch?v=ZtJqGx6fX8I. You'll notice you only see the artifacts when the homescreen is scrolled all the way up (which makes the statusbar transparent). If you scroll down a little (which makes the statusbar opaque black), the glitching doesn't appear. Perhaps it's unrelated to the statusbar color, but that's my best guess at this point.

(In reply to Benoit Girard (:BenWa) from comment #13)
> (In reply to Michael Henretty [:mhenretty] from comment #8)
> > After much investigation and profiling, I'm pretty sure this is a graphics
> > issue.
> 
> Can you list what you tried on the bug and your though process for coming to
> this conclusion. To me it doesn't feel like a graphics issue but perhaps if
> you shared your work so far that would help.

Basically, I narrowed down the code we called when closing lockscreen to this line [1] being the problem, and only when the statusbar is transparent on the homescreen. Furthermore, you only see the artifacting when the FPS counter is on, otherwise it is just a black background flashing like the original video.

1.) https://github.com/mozilla-b2g/gaia/blob/78be6285b44310cc45fd3c4dd41bb2e71bb7ce69/apps/system/js/system_dialog.js#L177

> Can we get a testcase reduction of that? Trying to simplify the app and
> system down to a minimal set of feature/complexity will help narrow down the
> bug. Particularly since the bug is very short lived debugging is more
> difficult.

I'm not sure how to narrow it down more than it being line [1] when the lockscreen is closing. It appears to be some interaction between lockscreen, statusbar and this dialog (all of which live in the system app). Note, it seems that if any other apps are open when the dialog displays, we don't have this issue.


> > After
> > systematically disabling js that runs on 'system-resize' using the profiler
> > as a guide, I fairly certain this is something lower.
> 
> Can you share your profile and what you saw in it that makes you think that
> so that I can follow your analysis? Can you show me what you disabled?

The profiler was only used to help me find the line in gaia which causes the problem when unlocking the screen. Here is a commit that demonstrates the narrowed down case, perhaps that will help:
https://github.com/mikehenrty/gaia/commit/698ead8595ba28ddb008ef5eec71fad5700cc753


> > BenWa, could you help me investigate this issue? The line that causes the
> > flashing/artifacting is when we update the height of the SIM PIN dialog here
> > [2]. But it only happens when closing the lockscreen when the statusbar is
> > transparent. Let me know if I can collect any more information here of use
> > to you.
> > 
> > 
> > 1.) https://www.youtube.com/watch?v=tmjGR6uyClI
> > 2.)
> > https://github.com/mozilla-b2g/gaia/blob/
> > 78be6285b44310cc45fd3c4dd41bb2e71bb7ce69/apps/system/js/system_dialog.js#L177
> 
> Can someone confirm if this is a regression? If so we should run a
> regression window. I'd be particularly curious to know if the offender was
> bug 1085223 based on the video in [2] if the regression is recent. Even if
> this didn't work in 2.1 we can still run a regression along the gecko
> changes that we're made in 2.2 to see if a change broke it.

It will be pretty hard to have QA run a regression window on this, since the bug is only uncovered after bug 1082071 landed. Yeah, I am also not convinced this is a regression at all. The only way I see is to bisect gecko with bug 1082071 applied, which I can probably help with, but I won't be able to get to it until later this week.
Flags: needinfo?(mhenretty)
(In reply to Milan Sreckovic [:milan] from comment #12)
> This only becomes 2.1+ if we actually uplift bug 1082071 to 2.1 - we should
> not do that and introduce a regression until we have the regression taken
> care of.  I've commented in bug 1082071 to that effect.  In the meantime, we
> can take a look at this on the trunk.  I have serious doubts that this is
> graphics, but Benoit can help find the problem.

We uplifted bug 1082071 to 2.1 last week :/
I just noticed that the log contains "y req scale factor beyond capability" which means this is likely the same underlying issue as bug 1075265.
See Also: → 1075265
I'm having trouble reproducing the bug on master. Going to setup a 2.2 build. Meanwhile since you can reproduce can you see if the bug goes away when you disable Hardware composer from the develop menu?

Notice how the glitches happens on the frames where the FPS counter doesn't render (those frames we use HWC)? This points to my theory in bug 1075265 Comment 12. Milan might be correct in bug 1075265 Comment 11. It could be that HWC has a scaling limitation that we don't check before running through HWC.
Flags: needinfo?(mhenretty)
Michael, while Sushil is away, can you find somebody to tell us if there are HWC limitations on the type of scaling it can do? Looking at some code it seems that anything outside of 8x scale (in either direction) may be a problem.
Flags: needinfo?(sushilchauhan)
Flags: needinfo?(mvines)
Alright I was able to reproduce the problem. Disabling HWC does in fact make the problem goes away. It looks like we're missing some checks in |gecko/widget/gonk/HwcComposer2D.cpp|. I've tried few obvious things (force a large scale) but that wasn't it. I'm going to dig deeper but this is something that codeaurora can probably give us a good answer on.
Sorry about the late response here. Yeah, turning off the HWC did indeed eliminate the issue. Also, this issue does happen on master for me, but it seems far less pronounced for some reason. Let me know if you want me to try anything else.
Flags: needinfo?(mhenretty)
So, Benoit, extending the check https://github.com/mozilla/gecko-dev/blob/master/widget/gonk/HwcComposer2D.cpp#L374 to see if the scale on the transform is < 1/8 or > 8 does not work?
Flags: needinfo?(bgirard)
(Diego, can you please check on this with folks here?)
Flags: needinfo?(sushilchauhan)
Flags: needinfo?(mvines)
Flags: needinfo?(dwilson)
(But yes there are limitations to the amount of scaling up/down the MDP can do.  1/4 to 8x seems about right, but I don't have the actual limits handy ATM)
Ok wow, I think we're hitting a ton of things here. First we need to fix to bug  1075265 but that's not sufficient. This further tweaks the opacity code.

I'm not certain why this is needed. It feels like the hardware composer touches the bits between a complex region so if we're not blending then we override the data. It would be nice to get someone who knows HWC more to comfirm that this is needed and a good fix. It fixes the graphical glitches for me.

We still get the black rendering which like I said earlier I suspect is a content bug (or layout bug at the worse).
Flags: needinfo?(bgirard)
Attachment #8513797 - Flags: review?(dwilson)
Regression caused by bug 981732.
Depends on: 981732
Hey Sotaro, do you feel comfortable reviewing this patch?

Note since Sushil is away and we're missing a bit of information we're going to be taking a bit of a risk putting this on 2.1.
Flags: needinfo?(sotaro.ikeda.g)
HwComposer hal's function optimizeLayerRects() seems a root cause of the problem.
https://www.codeaurora.org/cgit/quic/la/platform/hardware/qcom/display/tree/libhwcomposer/hwc_utils.cpp?h=b2g_kk_3.5#n1152
Flags: needinfo?(sotaro.ikeda.g)
optimizeLayerRects() do layer optimization without thinking about visibleRegionScreen. If layer is composed from multiple visible Region, it could expose un-rendered areas.
The patch fixes the problem at HwComposer level. It seems to fix the problem on local testing.
Comment on attachment 8516204 [details] [diff] [review]
patch - Do not optimize layers when layer has multiple visible regions

BenWa, can you give a feedback to the patch?
Attachment #8516204 - Flags: feedback?(bgirard)
Attachment #8513797 - Attachment is obsolete: true
Attachment #8513797 - Flags: review?(dwilson)
Comment on attachment 8516204 [details] [diff] [review]
patch - Do not optimize layers when layer has multiple visible regions

Review of attachment 8516204 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks Sotaro, this is a better fix!
Attachment #8516204 - Flags: feedback?(bgirard) → feedback+
Comment on attachment 8516204 [details] [diff] [review]
patch - Do not optimize layers when layer has multiple visible regions

Diego, can you review the patch? Now, it becomes clear the problem is caused by optimizeLayerRects() in /libhwcomposer/hwc_utils.cpp.
Attachment #8516204 - Flags: review?(dwilson)
When the HwComposer hal is landed, we could remove similar workaround from HwcComposer2D.
(In reply to Sotaro Ikeda [:sotaro] from comment #32)
> Comment on attachment 8516204 [details] [diff] [review]
> patch - Do not optimize layers when layer has multiple visible regions
> 
> Diego, can you review the patch? Now, it becomes clear the problem is caused
> by optimizeLayerRects() in /libhwcomposer/hwc_utils.cpp.

Sorry for the delay, I was on vacation. Have you checked if the video playback and camera preview cases still use HWC after this patch is applied?
Flags: needinfo?(dwilson) → needinfo?(sotaro.ikeda.g)
No problem. I confirmed that full screen video playback and camera preview are still using HwComposer by applying the patch on v2.1 flame.
Flags: needinfo?(sotaro.ikeda.g)
Comment on attachment 8516204 [details] [diff] [review]
patch - Do not optimize layers when layer has multiple visible regions

LGTM. Let me fix this to CAF
Attachment #8516204 - Flags: review?(dwilson) → feedback+
Thanks!
(In reply to Diego Wilson [:diego] from comment #36)
> Comment on attachment 8516204 [details] [diff] [review]
> patch - Do not optimize layers when layer has multiple visible regions
> 
> LGTM. Let me fix this to CAF

diego, is there a progress to CAF about this bug?
Flags: needinfo?(dwilson)
(In reply to Sotaro Ikeda [:sotaro] from comment #38)
> (In reply to Diego Wilson [:diego] from comment #36)
> > Comment on attachment 8516204 [details] [diff] [review]
> > patch - Do not optimize layers when layer has multiple visible regions
> > 
> > LGTM. Let me fix this to CAF
> 
> diego, is there a progress to CAF about this bug?

I landed this patch internally but the release process is stuck right now. I'll update you once it's in CAF.
Flags: needinfo?(dwilson)
diego, if we want to use the above change, which version(tag) should we use? LNX.LF.3.5.1_RB1.2?

current flame build use b2g_kk_3.5.

https://github.com/mozilla-b2g/b2g-manifest/blob/master/flame-kk.xml#L24
Flags: needinfo?(dwilson)
Whiteboard: [2.2-Daily-Testing] [shb-enabled] → [2.2-Daily-Testing][shb-enabled][POVB]
(In reply to Diego Wilson [:diego] from comment #40)
> Landed on CAF
> 
> https://www.codeaurora.org/cgit/quic/la/platform/hardware/qcom/display/
> commit/?h=LNX.LF.3.5.1_RB1.2&id=db3737d957e10661b3f8d2fb7e70e7fbda25a699

Just replace /hardware/qcom/display/ from revision="b2g_kk_3.5" to revision="LNX.LF.3.5.1_RB1.2" caused the build failure. MDSS_MDP_HW_REV_100 is already defined by out/target/product/flame/obj/KERNEL_OBJ/usr/include/linux/msm_mdp.h in moz build kernel source.

-----------------------------------------------------

target thumb C++: libqdutils <= hardware/qcom/display/libqdutils/mdp_version.cpp
In file included from hardware/qcom/display/libqdutils/mdp_version.cpp:31:0:
hardware/qcom/display/libqdutils/mdp_version.h:67:5: error: expected identifier before '(' token
hardware/qcom/display/libqdutils/mdp_version.h:67:5: error: expected '}' before '(' token
hardware/qcom/display/libqdutils/mdp_version.h:67:9: error: expected unqualified-id before numeric constant
hardware/qcom/display/libqdutils/mdp_version.h:67:9: error: expected ')' before numeric constant
hardware/qcom/display/libqdutils/mdp_version.h:67:9: error: expected ')' before numeric constant
hardware/qcom/display/libqdutils/mdp_version.h:67:9: error: expected ')' before numeric constant
hardware/qcom/display/libqdutils/mdp_version.h:67:9: error: expected ')' before numeric constant
In file included from hardware/qcom/display/libqdutils/mdp_version.cpp:31:0:
hardware/qcom/display/libqdutils/mdp_version.h: In member function 'bool MDPVersion::is8084()':
hardware/qcom/display/libqdutils/mdp_version.h:136:27: error: 'MDSS_MDP_HW_REV_104' was not declared in this scope
hardware/qcom/display/libqdutils/mdp_version.h: In member function 'bool MDPVersion::is8092()':
hardware/qcom/display/libqdutils/mdp_version.h:140:27: error: 'MDSS_MDP_HW_REV_206' was not declared in this scope
hardware/qcom/display/libqdutils/mdp_version.h: At global scope:
hardware/qcom/display/libqdutils/mdp_version.h:164:1: error: expected declaration before '}' token
make: *** [out/target/product/flame/obj/SHARED_LIBRARIES/libqdutils_intermediates/mdp_version.o] Error 1
moz build msm_mdp.h is very different compared to recent caf code. It seems to cause the problem.
Flags: needinfo?(dwilson)
Diego, is it possible to apply the fix to caf b2g_kk_3.5.
Flags: needinfo?(dwilson)
(In reply to Sotaro Ikeda [:sotaro] from comment #44)
> Diego, is it possible to apply the fix to caf b2g_kk_3.5.

Hey Sotaro, you probably should fork and apply the fix yourself in your tree.
clear need info. This is moz build specific problem.
Flags: needinfo?(dwilson)
Is this fixed on Mozilla's end at this point for v2.1+?
status-b2g-v2.1: --- → ?
Flags: needinfo?(sotaro.ikeda.g)
(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #47)
> Is this fixed on Mozilla's end at this point for v2.1+?

When we do shallow flash, the problem is fixed. But when we do deep flash, the problem exits on flame.
Flags: needinfo?(sotaro.ikeda.g)
(In reply to Michael Vines [:m1] [:evilmachines] from comment #45)
> (In reply to Sotaro Ikeda [:sotaro] from comment #44)
> > Diego, is it possible to apply the fix to caf b2g_kk_3.5.
> 
> Hey Sotaro, you probably should fork and apply the fix yourself in your tree.

mwu, should we create a fork by ourselves and fix the problem only for deep flash case? As in comment 43, just update related module to latest caf code causes the build failure on moz build.
Flags: needinfo?(mwu)
I think we should actually be switching to LNX.LF.3.5.1_RB1.2 where appropriate. We should be tracking CAF closely on Flame. I don't know if T2M is switching to these new releases yet though.
Flags: needinfo?(mwu)
Viral, do you know if we're getting a kernel source update for v10D?
Flags: needinfo?(vwang)
(In reply to Michael Wu [:mwu] from comment #51)
> Viral, do you know if we're getting a kernel source update for v10D?
Hi Michael,

So far t2m provide the latest release is v18D.
they still use caf b2g_kk_3.5 (tag:caf_AU_LINUX_GECKO_B2G_KK_3.6.01.04.00.000.120) for kernel

Shall we merge the fix in comment 40 to our own kernel branch (b2g/t2m-flame-3.4-kk)?
or we should move to LNX.LF.3.5.1_RB1.2 directly?
(i'm not sure t2m would like to change kernel branch in their side)
Flags: needinfo?(vwang)
Blocks: 1170825
Attachment #8643302 - Flags: review?(mwu)
Attachment #8643302 - Flags: review?(mwu) → review+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: