Closed Bug 675539 Opened 13 years ago Closed 5 years ago

Automatically unload (discard/hibernate) longly unused tabs to free resources / when running out of memory

Categories

(Firefox :: Tabbed Browser, defect)

defect
Not set
normal
Points:
13

Tracking

()

RESOLVED FIXED
Firefox 67
Fission Milestone M1
Tracking Status
firefox67 --- fixed

People

(Reporter: b1437400, Assigned: gsvelto)

References

(Blocks 3 open bugs)

Details

(Keywords: memory-footprint, Whiteboard: [Snappy:P2])

Attachments

(3 files, 5 obsolete files)

Since bug 586068 landed - there is a new state for every tab: it can be stalled (unloaded).

My suggestion is to automatically stall/unload tabs that were inactive for x hours in order to free some RAM.
The way I see it - there should be a pref in about:config which will define the time that needs to pass, before the tab (if it wasn't activated during that time) becomes stalled.
The same thing does BarTab addon.
Depends on: 586068
It may be that bug 591775 is the preferred method of achieving the desired outcome (freeing RAM) but hard to say if this bug is a dupe of that one.
Well, my bug is more particular (it's about tabs, no matter where they are).

I don't use tab groups at all, as it's not comfortable from my point of view.
I don't use multiple browser's windows. I use just one, and all the tabs are inside the single default group in it.
Sometimes I have ~100 or more tabs opened.
I use Tree Style Tab, so it's not a problem for me to manage such amount of tabs properly.

Fixing the bug you mentioned - won't make my browsing anyhow more comfortable, but fixing this bug I've created - will make everyone from the bug you mentioned plus me (actually plus others who browse the same way I do).
Close bug 591775 in favor of this bug?
See Also: → 591775
(In reply to comment #2)
> Well, my bug is more particular (it's about tabs, no matter where they are).
* I meant my bug is more general
Assignee: nobody → tim.taubert
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Attached patch patch v1 (WIP) (obsolete) — Splinter Review
That's a first WIP patch. It works as expected and tests pass locally. Some comments and thoughts:

>+    // observe the user's idle time (e.g. for tab hibernation)
>+    IdleSvc.addIdleObserver(this, 10);

After 10 seconds of idle time _hibernateInactiveTabs() is called and will hibernate inactive tabs. 10 seconds is probably too low and I don't know if we want to have that triggered purely on user idle time. I just figured it might be a good way to not rely on timers that could fire when playing your favorite html5 game.

>+  _hibernateInactiveTabs: function sss__hibernateInactiveTabs() {
>+    let self = this;
>+
>+    this._forEachBrowserWindow(function (aWindow) {
>+      Array.forEach(aWindow.gBrowser.tabs, function (aTab) {
>+        // don't hibernate visible or restoring tabs
>+        if (aTab.hidden && !aTab.linkedBrowser.__SS_restoreState)
>+          self._hibernateTab(aTab);
>+      });
>+    });
>+  },

1) We should of course only hibernate if the tab hasn't been shown for some time (probably hours). I think we should make this value configurable.

2) We could also hibernate tabs that are "visible" but haven't been selected for some time. I guess there are lots of users that have like 15-20 tabs open but don't use Panorama or Tree Style Tabs or the like. That could be particularly useful in conjunction with max_concurrent_tabs=0. We could also add a pref for this ;)

3) I have a big README group that contains maybe 30 tabs. If I look at one of those and switch to another group all of those 30 tabs will later be hibernated at once. Maybe we should split this process into chunks and say that we stop after e.g. 10 tabs were hibernated. We could also hibernate them one-by-one with setTimeout() like restoreHistoryPrecursor() does.
Attachment #550376 - Flags: feedback?(paul)
(In reply to comment #4)
> That's a first WIP patch. It works as expected and tests pass locally. Some
> comments and thoughts:

Try-server builds, please?

> After 10 seconds of idle time _hibernateInactiveTabs() is called and will
> hibernate inactive tabs. 10 seconds is probably too low and I don't know if
> we want to have that triggered purely on user idle time. I just figured it
> might be a good way to not rely on timers that could fire when playing your
> favorite html5 game.

If you play that game for hours - you'd probably don't need any other tabs to be loaded, so hibernating them is OK, IMO.

> 1) We should of course only hibernate if the tab hasn't been shown for some
> time (probably hours). I think we should make this value configurable.

Yeah, just as I requested.

> 2) We could also hibernate tabs that are "visible" but haven't been selected
> for some time. I guess there are lots of users that have like 15-20 tabs
> open but don't use Panorama or Tree Style Tabs or the like. That could be
> particularly useful in conjunction with max_concurrent_tabs=0. We could also
> add a pref for this ;)

That's the main idea of this bug as distinct from bug 591775 (which should be now marked as duplicate of this one, as you decided to fix this bug and it covers the same cases as bug 591775 does + more).
As for max_concurrent_tabs=0 - I think it should work only in case you opened the browser. Or maybe in case you switch to another group in Panorama (a pref for changing that behavior - would be nice).

> 3) I have a big README group that contains maybe 30 tabs. If I look at one
> of those and switch to another group all of those 30 tabs will later be
> hibernated at once.

And what's wrong with that behavior? If you didn't open any of those 30 tabs for a few hours - that's ok that they get stalled.

> Maybe we should split this process into chunks and say
> that we stop after e.g. 10 tabs were hibernated.

Why? Who'll decide which 10 tabs would be chosen and which stay loaded? Why 10? Why not 100?
I don't understand this point. What's for do you need this stop?

> We could also hibernate
> them one-by-one with setTimeout() like restoreHistoryPrecursor() does.

I'm not a programmer, I just think they should hibernate by timer.
If you are afraid of the moment that if 30 tabs would hibernate at once and that may cause some lag - well, yes, you might add some kind of a limit to slow down this process, like "wait 5 sec since last hibernation process started".
Summary: Automatically unload (stall) longly unused tabs to free RAM → Automatically unload (stall/hibernate) longly unused tabs to free RAM
(In reply to comment #5)
> (In reply to comment #4)
> Try-server builds, please?

Oh, I didn't push it to try, yet. Which platform are you on? Be reminded that hidden tabs are hibernated after about 10 secs idle time. You can check that with about:memory.

> > After 10 seconds of idle time _hibernateInactiveTabs() is called and will
> > hibernate inactive tabs. 10 seconds is probably too low and I don't know if
> > we want to have that triggered purely on user idle time. I just figured it
> > might be a good way to not rely on timers that could fire when playing your
> > favorite html5 game.
> 
> If you play that game for hours - you'd probably don't need any other tabs
> to be loaded, so hibernating them is OK, IMO.

True. Well, I guess we could also install some timer that fires regularly in small intervals and just hibernates a small amount of tabs. I kind of don't like that because it reminds me of our current GC pauses :)

> > 2) We could also hibernate tabs that are "visible" but haven't been selected
> > for some time. I guess there are lots of users that have like 15-20 tabs
> > open but don't use Panorama or Tree Style Tabs or the like. That could be
> > particularly useful in conjunction with max_concurrent_tabs=0. We could also
> > add a pref for this ;)
> 
> That's the main idea of this bug as distinct from bug 591775 (which should
> be now marked as duplicate of this one, as you decided to fix this bug and
> it covers the same cases as bug 591775 does + more).

Well the underlying technique is of course the same. But Panorama needs to be adapted to this new behavior and that is what I want to track in bug 591775. That's why it's blocked by this one.

> > 3) I have a big README group that contains maybe 30 tabs. If I look at one
> > of those and switch to another group all of those 30 tabs will later be
> > hibernated at once.
> 
> And what's wrong with that behavior? If you didn't open any of those 30 tabs
> for a few hours - that's ok that they get stalled.

Sorry I wasn't clear about that point. I'm actually afraid of the browser being stalled while hibernating 30+ tabs. That's why I suggested to hibernate them in chunks.
(In reply to comment #6)

> Which platform are you on?

Win7. Can also test in on WinXP if you like.

> Be reminded that hidden tabs are hibernated after about 10 secs idle time.

Ok, that's just for tests, anyways.

> True. Well, I guess we could also install some timer that fires regularly in
> small intervals and just hibernates a small amount of tabs. I kind of don't
> like that because it reminds me of our current GC pauses :)

I don't think it's needed.

> Well the underlying technique is of course the same. But Panorama needs to
> be adapted to this new behavior and that is what I want to track in bug
> 591775. That's why it's blocked by this one.

Oh, now I see.

> Sorry I wasn't clear about that point. I'm actually afraid of the browser
> being stalled while hibernating 30+ tabs. That's why I suggested to
> hibernate them in chunks.

Chunks is not really a good idea in that case, IMO. On slow PCs even 2 parallel hibernating process might cause some lags (especially when you play a HTML5 game, which may get affected like with a decreased FPS rate). That's why I suggested this:
> If you are afraid of the moment that if 30 tabs would hibernate at once
> and that may cause some lag - well, yes, you might add some kind of a
> limit to slow down this process, like "wait 5 sec since last hibernation
> process started".
If we are talking about timers set to approximately a few hours - I think the 5 minutes plus/minus don't matter much, so maybe it's better try to use a queue for that?
Well, if you like to get n tabs simultaneously go to stall - you could add a pref like "max_hibernating_tabs", but taking in consideration the users with slow PCs - then I'd suggest to use 1 as default value for that.
Tim, since the patch is WIP and it's the best moment to apply changes to it - may I ask you - how did you implement this feature?
Did I get you right, that when a tab hibernates - it gets put to cache and when it gets activated later - it reloads from cache, not from the internet, right?
If so - I foresee that some people might want to have another behavior: at hibernation - the tab's info gets deleted, and when it gets activated - the page loads from the internet.
Could you add a pref for that?
(In reply to comment #8)
> Tim, since the patch is WIP and it's the best moment to apply changes to it
> - may I ask you - how did you implement this feature?

The tab info is just saved like when you quit Firefox. And the same tab restore mechanism is applied when this tab gets activated again. So it's the same as "Show my windows and tabs from last time". Form data and history entries are saved. The url is just loaded again from the internet, from the cache if the web page itself allows us to.
(In reply to comment #7)
> (In reply to comment #6)
> > Which platform are you on?
> Win7. Can also test in on WinXP if you like.

Here's the build:

https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tim.taubert@gmx.de-6a8c230f43cc/try-win32/
(In reply to comment #10)
> Here's the build:

Not working for me at all.
(In reply to comment #11)
> (In reply to comment #10)
> > Here's the build:
> Not working for me at all.

This works only with hidden tabs, when using Panorama.
Oh, just re-tested it and it works good.
(In reply to comment #13)
> Oh, just re-tested it and it works good.

- Well, actually not good enough: sometimes it doesn't work. I have a feeling that it works only in case the user shows absolutely 0 activity, because if I do something while waiting the 20sec timer (I know it's 10sec) and then switch to another group - tabs are not hibernated in that group.
But if I don't touch anything and just wait ~10sec - the tabs from other groups actually hibernate.
- every time some tab hibernates - it loses it's favicon (it's not shown on tab bar and in Panorama), could you please try to change that behavior?
Panorama

+ I like that the thumbnail of a hibernated tab doesn't get erased, and it remains in Panorama.
(In reply to comment #14)
> - Well, actually not good enough: sometimes it doesn't work. I have a
> feeling that it works only in case the user shows absolutely 0 activity,
> because if I do something while waiting the 20sec timer (I know it's 10sec)
> and then switch to another group - tabs are not hibernated in that group.
> But if I don't touch anything and just wait ~10sec - the tabs from other
> groups actually hibernate.

That's how the idle service works. It checks if the user is really idle (not only in Firefox). That may not be the best solution but normally you won't wait for your tabs to hibernate :)

> - every time some tab hibernates - it loses it's favicon (it's not shown on
> tab bar and in Panorama), could you please try to change that behavior?
> Panorama

I'm gonna tackle this in bug 591775 after this one is done. Or probably after I know what this is gonna look like.
(In reply to Tim Taubert [:ttaubert] from comment #15)
> That's how the idle service works. It checks if the user is really idle (not
> only in Firefox). That may not be the best solution but normally you won't
> wait for your tabs to hibernate :)

This is bad and useless :(
So, if I play a HTML5 game for a few hours - zero tabs will get stalled. Then it becomes a pretty useless feature.
What's the point of such behavior? the tab should hibernate if _exactly_this_tab_ is inactive for a few hours.
Yes, even the bug title says 'tabs that are unused are hibernated'. Not when the user is idle or computer is unused - that is a subset of the wanted behavior. The memory should be freed from unused tabs because the user is active and needs it for other tabs.
Attached patch patch v2 (obsolete) — Splinter Review
New patch, new approach:

Every 'n' seconds (defined by browser.sessionstore.hibernate_interval) we check for tabs that can be hibernated. This depends on whether we restore on-demand and whether we restore hidden tabs. If the tab has not been selected for 'x' seconds (defined by browser.sessionstore.hibernate_after) it will be hibernated.

Some facts:

1) I introduced two WeakMaps called _windowDataStore and _tabDataStore. These take a <tab> or <window> as key and return a plain old javascript object to store data in. I needed those for this patch but I could of course have added another property like "__SS_*". So dunno if you like that approach but maybe you could use that in session store rather than polluting the object's properties. I'll definitely have look at that for Panorama...

2) Tabs are hibernated in chunks. That means we'll hibernate 10 tabs max. when ._hibernateInactiveTabs() runs. Remaining tabs get hibernated in subsequent intervals. That seemed like a good idea to me to not lock up the UI as long as we keep the interval reasonably small.

3) Tabs are hibernated using setTimeout() for the same reason as (2).

I'll write a test if you like that approach :)
Attachment #550376 - Attachment is obsolete: true
Attachment #554220 - Flags: feedback?(paul)
Attachment #550376 - Flags: feedback?(paul)
(In reply to Tim Taubert [:ttaubert] from comment #18)
> New patch, new approach:

A try-server builds please? [at lease for Win7x32] 

> Every 'n' seconds (defined by browser.sessionstore.hibernate_interval) we
> check for tabs that can be hibernated. This depends on whether we restore
> on-demand and whether we restore hidden tabs. If the tab has not been
> selected for 'x' seconds (defined by browser.sessionstore.hibernate_after)
> it will be hibernated.

Why not make "n" "set to stone" equal 1 sec? Will it cause any lag?

> 2) Tabs are hibernated in chunks. That means we'll hibernate 10 tabs max.
> when ._hibernateInactiveTabs() runs. Remaining tabs get hibernated in
> subsequent intervals. That seemed like a good idea to me to not lock up the
> UI as long as we keep the interval reasonably small.

But why not use a chain instead of chunks? We won't have that limit of 10 tabs per one "n" interval time out and the interval itself. And if we stall only 1 tab at the same time - it's the most optimal lagless situation.

> I'll write a test if you like that approach :)

Actually I don't like that approach. I still don't understand why don't you like the one suggested by me, to use query and stalling tabs as a chain reaction.
(In reply to Joe Wilson from comment #19)
> A try-server builds please? [at lease for Win7x32]

https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/tim.taubert@gmx.de-3b928fc28dff/

Windows builds aren't ready, yet. As soon as they are you will see the folder appear.

> Why not make "n" "set to stone" equal 1 sec? Will it cause any lag?

I think that would work but I don't see a reason to do this. It's totally sufficient to check once every minute for tabs tab have been inactive for like two hours. Hibernating tabs is not at all a time critical operation and we should really do this in the background. Maybe even a 60 secs interval is bit too often. Though, you can play with it and adjust the prefs.

> But why not use a chain instead of chunks? We won't have that limit of 10
> tabs per one "n" interval time out and the interval itself. And if we stall
> only 1 tab at the same time - it's the most optimal lagless situation.

I don't quite get what the difference between my "chunk" and your "chain" is. If you think that we should query every second for inactive tabs and hibernate only one of them, then after a sec query the next and hibernate one again, then this is a rather similar approach but with and interval of 1 sec and a chunk size of 1 - this is not any different.

> > I'll write a test if you like that approach :)
> Actually I don't like that approach.

That was directed at Paul and I don't mean the implementation in every detail but rather the overall technical approach.
Note that "hibernating" can cause dataloss, esp. when plugins are involved. Obviously this is already an issue with session restore, but it's less severe there since it doesn't happen in the middle of using the browser without the user's consent.
Comment #8:
>If so - I foresee that some people might want to have another
>behavior: at hibernation - the tab's info gets deleted, and when
>it gets activated - the page loads from the internet.

I'm skeptical about the safety of reloading idle tabs without user consent. If tabs timeout and are unloaded without user intervention and are then reloaded from the Internet without user intervention, essentially the browser will be executing a reload operation without user consent.  This could have bad side effects for ecommerce sites (such as repeated transactions) and other sites (such as forums and blog comment pages) that consider repeat page hits or reloads significant.  It would not be desirable for a user to leave their browser on an ecommerce page that adds $5,000 widgets to their shopping cart and return to find that FF's hypothetical hidden reloading means that they have more widgets than they want.

If FF unloads unused tabs to save memory, then the process must be essentially transparent to the user (e.g. a delay when switching to an unloaded tab is acceptable) but *completely* transparent to the web server.  Nothing should be sent to the server to suggest that anything has happened to the user's view of the page.  At minimum, neither PUT nor GET requests should be repeated, but ideally nothing at all should be transmitted to the server when an inactive tab is brought back into memory.

Comment #21:
> Note that "hibernating" can cause dataloss, esp. when plugins
> are involved. Obviously this is already an issue with session
> restore, but it's less severe there since it doesn't happen in
> the middle of using the browser without the user's consent.

A short term solution would be to block tab unloads on pages that use plugins that are not specifically known to be stateless. Coordination might be needed with anti-plugin extensions (such as noscript and adblock) to ensure that pages where plugins have been denied permission to load are still considered unloadable.

A longer term solution would be to provide a mechanism for plugins to communicate state data with the browser so that session restore and tab unloading can be completely lossless.
(In reply to ch-bugzilla from comment #22)
> I'm skeptical about the safety of reloading idle tabs without user consent.

I said "SOME people MIGHT want to have it that way". So I definitely didn't mean it to be a default behavior. I'm just reminding that having a choice is always nice.

> A longer term solution would be to provide a mechanism for plugins to
> communicate state data with the browser so that session restore and tab
> unloading can be completely lossless.

I think a longer term solution would be to wait until e10s lands, so we could just not kill the plugin's process at all.
Contents of forms, scrollbar positions, cookies, and POST data would all stay?

Because with session restore that's not necessarily the case (controlled by the browser.sessionstore.privacy[_deferred] settings).
(In reply to [Baboo] from comment #24)
> Contents of forms, scrollbar positions, cookies, and POST data would all
> stay?
> 
> Because with session restore that's not necessarily the case (controlled by
> the browser.sessionstore.privacy[_deferred] settings).

Resuming a hibernated works the same way as restoring a tab after a browser restart (via session [re]store) - so the same prefs should apply.
Isn't that an dataloss issue then when the applying setting (browser.sessionstore.privacy or browser.sessionstore.privacy_deferred?) is set to anything but 0?
A hibernation with dataloss must only be enabled if the user wants it.
The default hibernation method should be without dataloss and without network reload. Something like storing the whole page state (DOM tree, etc) into a cache file and restore from that. Also load all other linked files from cache (not from network).
(In reply to [Baboo] from comment #26)
> Isn't that an dataloss issue then when the applying setting
> (browser.sessionstore.privacy or browser.sessionstore.privacy_deferred?) is
> set to anything but 0?

Yeah, that would be data loss :|

(In reply to aceman from comment #27)
> A hibernation with dataloss must only be enabled if the user wants it.
> The default hibernation method should be without dataloss and without
> network reload. Something like storing the whole page state (DOM tree, etc)
> into a cache file and restore from that. Also load all other linked files
> from cache (not from network).

Seems like we would need a more sophisticated method of (re)storing a tab's state when hibernating. I agree, that automatic tab hibernation is a very different workflow than intentionally closing the browser.
Whiteboard: [MemShrink]
We discussed this in today's MemShrink meeting and decided that the safety-of-reloading and data-loss issues seem difficult enough that this approach probably isn't worth pursuing, especially if it wouldn't be a default option.  So I've removed the MemShrink whiteboard annotation.

However, there is plenty of non-critical state that can be discarded, and we could be more aggressive and organized about that.  I started a thread on dev-platform to discuss this:  http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/44a0efec8a336867#
Whiteboard: [MemShrink]
A short note to mention an experimental extension, as a workaround that achieves this while discussion continues, called Dormancy:
https://addons.mozilla.org/en-US/firefox/addon/dormancy/
Can you elaborate how the extension works? It is not described on the homepage. How does it free the memory? What state information is lost, if any? Does it reload the page from network when restoring? See comments 27-28.
I'm the author of the add-on. It basically does what this patch does, but in a less graceful way. It saves the serialized tab data on the tab itself, and sets the location of the tab to about:blank.

At first I didn't notice a big difference, but after some more testing, I found that it decreased memory use by up to 50% with very large tab sets. Caveat: I was just eyeballing about:memory with the add-on enabled/disabled.

We could really use some proper testing with this patch before being able to effectively weigh the costs vs the benefits of this approach.
Wrt to this feature in Firefox itself, I think we should check it into Nightly for a period of time, defaulted to off, with a preference to flip it on.

That will allow us to get broader feedback at very low cost.

The dev.platform thread is the right place to determine if there is a safety risk, and exactly how much might be lost in the suspend/restore cycle.

Dev.platform is not the right forum to decide whether *users* think the cost is worth the benefit.

In my experience, the tabs that get this treatment are rarely the critical ones wrt to data loss - eg, where you're interacting with forms, etc.
May I also suggest,while this change is been discussed,to consider adding the two BarTab features,"Unload tab" on demand from the tab context menu (very handy to instantly shut down tabs with obnoxious flash content,without actually killing them) and "Keep Tabs from URL Loaded ",which may also be beneficial to the data loss issue?
BarTab works so seamlessly in Firefox 3 (IMHO) that would be a shame to not have such a functionality built into the latest Firefox versions.
I think we should leave this feature to add-on, a add-on developer already do that: https://addons.mozilla.org/en-US/firefox/addon/dormancy/

I love this feature, but it may cause data loss like BarTab before ?
Memory problem can easy solve by adding a Restart menu on App Menu or File menu, restart app is the best way to release memory.
I am most interested in this not for reasons of RAM use but browser usability ("snappiness"). I want a browser that is snappy regardless of how many tabs are in the session. Unfortunately, for example with Bar Tab Lite currently, after a few hundred tabs (say 300-500) Firefox becomes very very slow and is prone to freezing up for, say, 2-10 seconds at a time. Mundane tasks like copying the URL of the current page into the clipboard take forever. Issues with regard to big session sizes need to be tackled within Firefox, not extensions.
A very easy way to observe the issue I'm describing is that when watching Flash video the whole browser freezes, with the audio continuing as normal but the video of course (as part of the tab) being affected.
I suppose I should check if my specific issue has been reported already.
I second the request for an "unload tab" option, or functionality similar to BarTab on Firefox 3 (with a exception list to prevent certain tabs from unloading). 

My apologies to Dietrich, but Dormancy is a joke (I tried it on Palemoon 7.0.1 x64 and it caused the loss of URL data in the session file, rendering most of the tabs unrecoverable).  BarTab Lite is also useless, as it never appears to free memory, as is Tab Utilities, which has the ability to "reload/restart" a tab after a given timeout, but it too does not appear to free memory.  FWIW, I use Palemoon/Firefox/Waterfox with Tree Style Tab.

If the Session Manager can create tabs without loading them ("unloaded tabs"), then I don't see why after a given timeout Firefox cannot replace an active tab with a something similar to what Session Manager creates, and free the tab's resources from memory.

Right now the only solution is to perform an application restart periodically.
Suggestion A:
A way to unload a tab that saves it to disk with all entered form info etc, and that when restoring does NOT ask the website for an updated version or perform anything else to reload it (that in UX terms feels exactly like tabbing - so none of the "quick refresh" you see when clicking a tab BarTab Lite has so far prevented from loading).
Suspend tab(s), write to disk, unload. Re-load on demand, resume.
Questions would be:
Can this be done in extension form (make it call Firefox' existing load tab from disk code while telling it not to ask the website if it may) or would it require code changes (code alteration/reimplentation)?
Would it be possible to save the state a plugin is in alongside? This couldn't work for, say, flash games that require permanent internet connections (Farmville), but surely for Winterbells.




Suggestion B:
Something more akin to Chrome's task manager than to about:memory to illuminate the user's understanding of how resource-intensive various websites he uses are, to help him decide if the Ajax stuff (and/or bad website coding) that enables realtime Twitter/Facebook/.. notifications, Envolve chat systems and so on are something he wants to "pay for" in performance or to suspend (see Suggestion A). This would need to show not only the current RAM and CPU power used by a tab but also the RAM use increase since initial loadup of the tab has finished as well as the average & total CPU time used ever since initial loadup.
Suggestion B is bug 400120.
(In reply to Shaun from comment #37)
> I second the request for an "unload tab" option, or functionality similar to
> BarTab on Firefox 3 (with a exception list to prevent certain tabs from
> unloading).

Bartab has been in fact patched to support FF4 and above. The original developer has abandoned the project so update is not available as installatable add-on. You can get buildable source from here:
https://github.com/fhoech/BarTab

It would be ideal to publish the fork as an add-on but at least I am vary of putting up the modified "binary" without author's consent. The one I made for myself works quite nice in FF8.

Just as an aside about caching the tabs to disk, firefox will consume enormous amounts of memory with a few hundred tabs. If you swap 1GB (yes!) worth of tabs to disk, that's a _lot_ of disk action. I suppose "caching" scheme would just save images and source as-is instead of the tab state, memory leaks and all.
Olli, you did check out BarTab Lite, yes? And looked at the source of that? ( https://github.com/philikon/BarTabLite )
And it's not hard to ask the author anything.. I presume http://twitter.com/#!/philikon or Facebook are fastest.
Yes, it refers to mozilla license which I'm not familiar with.. In any case the "lite" version doesn't do what bartab does. Twitter doesn't have personal message-function so I quess I'll just try facebook. 

In any case I'd prefer "bartab-tryout" is "published" by someone who actually contributed to the code. I certainly didn't have anything to do with it.
Very offtopic by now, but:

Mentioning somebody on Twitter ("@philikon, do you mind if...") will give you an email notification of a reply; at http://twitter.com/#!/messages you can send direct private messages.
(In reply to Shaun from comment #37)
> My apologies to Dietrich, but Dormancy is a joke (I tried it on Palemoon
> 7.0.1 x64 and it caused the loss of URL data in the session file, rendering
> most of the tabs unrecoverable).

No apologies necessary. Though I prefer to refer to it as an experiment, and many warnings were given in that blog post ;)

The experiment was a success IMO - it proved (in my case) the concept - saving a ton of memory for profiles that have many tabs open.
We could just CC philikon to the bug ;-)
Me love you long time.
This is just my about:memory page. Bartab (patched by fhoech et al) is running and has duly unloaded most tabs after 12 hour timeout. However it seems firefox is hanging on to a lot of that memory anyways with roughly 200MB sitting in "reserve"
(In reply to Olli Männistö from comment #47)
> This is just my about:memory page. Bartab (patched by fhoech et al) is
> running and has duly unloaded most tabs after 12 hour timeout. However it
> seems firefox is hanging on to a lot of that memory anyways with roughly
> 200MB sitting in "reserve"

That is bug 681201.
Who uses BarTab: have you ever had problem with bartab and tab groups. I've tried BarTab but when it disabled tab, it changed its tab groups. It made me a little bit of mismatch. I must disable BarTab. Actually I use BarTab Lite and it's not as good as full BarTab. A few minutes ago, I must kill Firefox with 50% cpu usage, and 1,5GB of ram (lots of tabs)....
I have never used tab groups so can't comment on it. UI seemed awfully user-hostile plus it didn't create thumbnails for some reason (bartab?) even when reloading all tabs.

WRT dataloss fears causing devs to abandon this line of improvement entirely, there is already an [url=https://addons.mozilla.org/en-US/firefox/addon/lazarus-form-recovery/?src=search]addon to preserve form data[/url]. Any reason why something like this could not be implemented for built-in bartab? If we adopt long time to unload tabs (say, 12 hours) the session should be expired anyways.
I've used both BarTab and BarTab Lite, although I've never used tab groups. Never had an issue with either extension.  My browsing experience is much worse without being able to hibernate tabs. I typically have 20 - 100 tabs open during any one browsing session.  I only use one window, too.  The more tabs I have open, the worse my browsing experience becomes... unless I use BarTab Lite (which no longer works as of FX 8, even if I bump the version in the .rdf before installing it).

Personally, I'd want to be able to configure the amount of time that expires before FX hibernates or unloads tabs.  Waiting 12 hours is way, way too long.  My thinking is to make it configurable, in minutes, like in increments of 5 or 10.  When I'm researching something, I'll have dozens of tabs open.  My machine is maxed out at 2 GB RAM.  When I have about 20 tabs open now, the browser becomes unbelievably sluggish because of the RAM all those open tabs require.  I don't care that tabs will have to reload.  At least I have all those tabs open I can return to when I need to access them.  As soon as I click on one that's been unloaded, it automatically loads.  Perfect.

One more thing... This is probably out of scope, but it would be fantastic if this functionality could apply to a grouping of loaded tabs.  Opera has a feature where you can expand or contract a group of tabs.  However, it doesn't unload the tabs so, very shortly, it consumes close to a gig of RAM and I have to shut it down.  It would be great if tab hibernation was implemented in such a way that it applied to a tab group, too.  Contract the group (a la Opera) and unload all the tabs.  Expand the group but let the user decide which of the tabs in the group to load by clicking on them (or make it an option to load them all automatically).

Hope all that made sense.
(In reply to mlp0219 from comment #51)
>
> 
> Personally, I'd want to be able to configure the amount of time that expires
> before FX hibernates or unloads tabs.  Waiting 12 hours is way, way too
> long.  My thinking is to make it configurable, in minutes, like in
> increments of 5 or 10.  

 I'll second that,a 12 hour time frame to unload inactive tabs would most of the times have no impact on the browser,it's not like all heavy tab users leave Firefox running for 12 hours or more. 

 > I don't care that tabs will have to reload.  At
> least I have all those tabs open I can return to when I need to access them.
> As soon as I click on one that's been unloaded, it automatically loads. 
> Perfect.

 Indeed,that's how BarTab used (still does,in 3.6.x) to work,and I've never had any issue.It works.Reloading when needed was no issue.
And,BTW,"unload tab(on demand)" and "keep tabs from www.etc loaded" added more control when needed.
(In reply to msth67 from comment #52)
>  I'll second that,a 12 hour time frame to unload inactive tabs would most of
> the times have no impact on the browser,it's not like all heavy tab users
> leave Firefox running for 12 hours or more. 

That's just my personal preference. You can adjust it in bartab and so should you be able to in any integrated FF functionality. You should, however, be clearly notified of the possibility of data loss and/or issues with resubmit of forms if you put in something like 30min.. Also WRT Lazarus, is saving form data really something completely unthinkable for core functionality??

>  Indeed,that's how BarTab used (still does,in 3.6.x) to work,and I've never
> had any issue.It works.Reloading when needed was no issue.
> And,BTW,"unload tab(on demand)" and "keep tabs from www.etc loaded" added
> more control when needed.

So go get the Bartab 3.0 build (philikon, feel free to do that pull request anytime..), [url=https://addons.mozilla.org/en-US/firefox/addon/bartab/]someone put an XPI to mediafire in the comment thread[/url]
I think the core is already storing typed in form data and many other state data of open tabs in the sessionstore file for crash recovery.
So form data can be stored in there, and all the other files (html, js, img) can be reloaded from cache. No need to reload from network. Yes, there may be sites that change the source (and DOM) via js, so the current state is nowhere in the cache. That is what would need to be serialized to disk too.
(In reply to mlp0219 from comment #51)
> When I'm researching something, I'll have dozens of
> tabs open.  My machine is maxed out at 2 GB RAM.  When I have about 20 tabs
> open now, the browser becomes unbelievably sluggish because of the RAM all
> those open tabs require.
Just as a small note, using RAM in and of itself shouldn't slow things down beyond increasing GC and CC times (which should be manageable). But if your machine starts swapping, that is using your hard drive to supplement the RAM, that will slow things down to a crawl.
(In reply to Emanuel Hoogeveen from comment #55)
> Just as a small note, using RAM in and of itself shouldn't slow things down
> beyond increasing GC and CC times (which should be manageable). But if your
> machine starts swapping, that is using your hard drive to supplement the
> RAM, that will slow things down to a crawl.

However in practise, firefox will start to crawl anyways when RAM usage goes above 2GB and it'll crash when it exceeds 3GB. In a 8GB machine that isn't even straining resources yet.
(In reply to Olli Männistö from comment #56)
> > However in practise, firefox will start to crawl anyways when RAM usage goes
> above 2GB and it'll crash when it exceeds 3GB. In a 8GB machine that isn't
> even straining resources yet.

I have 24GB of DDR3 in my main box, and I use the 64-bit Windows Nightly builds and after a full days use, Nightly is up to about 1.5GB of usage and performance is terrible. I don't care about high RAM usage, I care about the poor performance, so anything that can help offset this is welcome in my book.
I visited the BarTab addon page, https://addons.mozilla.org/en-US/firefox/addon/bartab/, and found that someone had posted a link to BarTab 3 on Mediafire.  I downloaded it, bumped the maxversion in its install.rdf to 9.0.* and installed it in 9.0b1.  As functional as ever w/ no problems... so far.  I have it configured to unload unused tabs after 10 secs and not to load tabs when restoring them from a previous session.  I can also unload tabs on demand via the context menu with no problem.  There's no way this functionality should not be included in base FX code.  That's just MNSHO.
@(In reply to mlp0219 from comment #58)
Thanks for the heads up!
BarTab 3 does appear to be working on Palemoon 8.0-x64.
Deepest thanks and kudos to whoever did the patch.
Ultra Cool!!!  I mean REALLY ultra cool...
Why this hasn't been integrated into the base code is also a mystery to me. (???)

That said, I'd still like to see it enhanced to leverage a cached state stored on disk instead of having to reload the tabs from scratch.  The one downside of using BarTab is that you do lose CGI Post result pages like StartPage/ixquick search results, unless you list them sites as exceptions that should never be unloaded.  Doing so is easy to do using the tab context menu, but you need to remember to do so.  :)

My migration away from FF3 is now FINALLY complete.  :)
(In reply to Brian Carpenter [:geeknik] from comment #57)
> I have 24GB of DDR3 in my main box, and I use the 64-bit Windows Nightly
> builds and after a full days use, Nightly is up to about 1.5GB of usage and
> performance is terrible. I don't care about high RAM usage, I care about the
> poor performance, so anything that can help offset this is welcome in my
> book.

that's not this bug then. can you file a new bug with your system details, list of add-ons, etc, and cc me? we should dig further into exactly where the bottlenecks are in your configuration.
It would also be very nice to have an "automatically unload unused tabs" function not just to free up RAM, but also reduce the possible CPU usage which may increase battery life. Somehow after u/g to beta version of firefox, considerably high CPU usage frequently occurred. Though I'm not sure if it's due to some webpage or addons. Would be nice to have a good way to check. (I used to keep ~1-200 tabs opened)

top - 23:57:00 up 3 days, 49 min,  9 users,  load average: 0.77, 0.75, 0.85
Tasks: 230 total,   2 running, 228 sleeping,   0 stopped,   0 zombie
Cpu(s): 19.0%us,  2.7%sy,  0.0%ni, 78.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3911096k total,  3383200k used,   527896k free,    65012k buffers
Swap:  9896000k total,        0k used,  9896000k free,  1301600k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
24468 xxxx      20   0 1818m 1.3g  41m S    8 34.4 206:06.69 firefox            
 1197 root      20   0 93172  62m  17m S    2  1.6  67:10.64 Xorg               
 1619 xxxx      20   0 82516  20m  13m S    2  0.5   1:57.66 gnome-terminal
Comment on attachment 554220 [details] [diff] [review]
patch v2

Review of attachment 554220 [details] [diff] [review]:
-----------------------------------------------------------------

Well this slipped off my radar :(

There are valid concerns about losing data and this won't be perfect, but we do trust it for restoring post-update & post-addon-install (and after every startup) so it's probably good enough. It will cause a problem for somebody, but it's off by default (though a bunch of people have turned "on demand" restore on, sooooo maybe we need another switch)

::: browser/components/sessionstore/src/nsSessionStore.js
@@ +411,5 @@
>      }
>  
> +    // initialize data stores for windows and tabs
> +    this._windowDataStore = new WeakMap();
> +    this._tabDataStore = new WeakMap();

I applaud the use of weakmaps here, but I don't think we need it. The data stores are mostly just for tracking timestamps which I've wanted to do in sessionstore anyway, so let's do it. Go go gadget bug 586067.

I guess windowDataStore is used to track the previous/current selected tab, but we already have that (essentially): this._windows[aWindow.__SSi].selected (in onTabSelect) is the index.

@@ +691,5 @@
> +      case "sessionstore.hibernate_interval":
> +        this._hibernateInterval =
> +          this._prefBranch.getIntPref("sessionstore.hibernate_interval");
> +        if (this._shouldHibernateInactiveTabs())
> +          this._startHibernateTimer();

This isn't going to actually update the interval if the timer is running is it? That checks for _hibernateTimer and does nothing if it already exists.

@@ +1220,5 @@
> +      // update timestamp for the previously selected tab
> +      if (winData.lastSelectedTab) {
> +        let tabData = this._tabDataStore.get(winData.lastSelectedTab);
> +        tabData.lastSelectedAt = Date.now();
> +      }

It's not really a lastSelectedAt is it, but when it was last visible. Not sure what it should be called in that case.

@@ +4210,5 @@
> +    if (!this._hibernateInterval || !this._hibernateAfter)
> +      return false;
> +
> +    // all tabs are always restored - so there's nothing to hibernate
> +    return (this._restoreOnDemand || !this._restoreHiddenTabs);

return !(_hibernateInterval && _hibernateAfter) && (_restoreOnDemand || ! _restoreHiddenTabs)
(but prettier)

@@ +4221,5 @@
> +    let tabsToHibernate = [];
> +
> +    this._forEachBrowserWindow(function (aWindow) {
> +      tabsToHibernate.push.apply(tabsToHibernate, aWindow.gBrowser.tabs);
> +    });

I think it would make things more readable to check _shouldHibernateTab here as opposed to just adding every tab ever and checking in hibernateNextTab. Oh but if the condition of the tab changes you want to know... in that case tabsToHibernate is just misleading currentTabs, allTabs, something along those lines makes more sense.

@@ +4234,5 @@
> +        self._hibernateTab(tab);
> +        numHibernated++;
> +      }
> +
> +      if (numHibernated < 10 && tabsToHibernate.length) {

Put 10 in a constant up top, not squirreled away like this.

@@ +4262,5 @@
> +
> +    // don't hibernate if the tab would be restored immediately
> +    if (!aTab.hidden && !this._restoreOnDemand)
> +      return false;
> +

Nit: a lot of `return false` happening... I want to combine them & have a detailed comment above the if blockN
Attachment #554220 - Flags: feedback?(paul) → feedback+
I apologize in advance if this may sound all too obvious,but is there any telemetry-related study for profiles with BarTab installed (as far as I know,the last compiled version works perfectly on the nightlies,I'm currently using it)vs. profiles without it ? 
I ask because,from my non-technical point of view,it makes such a positive impact on the browser's performance and overall responsiveness that there has to be some useful data somewhere:for a similar heavy usage (around 100 tabs open,with lots of images)without BarTab the browser slows down so much (really *bad* scrolling,huge delays on tab opening/closing,general loss of responsiveness)that it has to be periodically restarted,with BarTab an acceptable level of performance is retained even in the most critical situations.
(In reply to Tim Taubert [:ttaubert] from comment #9)

> The tab info is just saved like when you quit Firefox. And the same tab
> restore mechanism is applied when this tab gets activated again. So it's the
> same as "Show my windows and tabs from last time". Form data and history
> entries are saved. The url is just loaded again from the internet, from the
> cache if the web page itself allows us to.

Oh, please change that. The hibernation feature would be really great, but it has to be transparent to the user. So data entered in forms are saved and restored. Similarly, the page *as it was* has to be saved and restored. Not reloaded from the Internet. Think about pages which quickly disappear or change on the Internet. Think about transaction / webmail / forum posting / prefs changing / bank transfer / shopping / payment pages.

I suggest storing the hibernation in a dedicated cavern rather than using the general cache.

Let's imagine this scenario :

In one tab, I am editing Wikipedia. I have written some stuff and I have clicked “Preview”. My text is not posted yet, I am previewing and tweaking it. In another tab, I am writing in a forum, I am in the same “Preview” step. For whatever reason, I empty the cache. Later, I go to lunch, I am outside, several hours pass. When I come back to my computer, I find the Wikipedia tab exactly as I had left it, I continue previewing and tweaking my text, and I click “Save page”, and my text is posted on Wikipedia. The same goes for the forum tab.

Now, with the hibernation feature, this scenario should keep working flawlessly. While I am outside, my Firefox tabs go to sleep. When I come back to them, they wake up, and I find them in the same state as they were. I certainly don't want to find them broken because Firefox tried to restore them from the cache and the cache had been emptied in the meantime.

Furthermore, some contents get displayed but not stored in the cache because the server tags these contents as such.

Furthermore, I think of the Firefox Portable, who has its cache set to “0 MB” if I remember correctly.

Thanks.

(In reply to Tim Taubert [:ttaubert] from comment #20)

> It's totally
> sufficient to check once every minute for tabs tab have been inactive for
> like two hours. Hibernating tabs is not at all a time critical operation and
> we should really do this in the background. Maybe even a 60 secs interval is
> bit too often. Though, you can play with it and adjust the prefs.

Sure. Even once every 10 minutes would be more than enough. Let's not have the performance-improving feature eat the performance. ;-)
Keywords: footprint
Whiteboard: [Snappy]
Whiteboard: [Snappy] → [Snappy:P2]
Attached patch patch v3 (obsolete) — Splinter Review
Overhauled the whole patch and filed bug 747338.
Attachment #554220 - Attachment is obsolete: true
Attachment #616928 - Flags: review?(paul)
Comment on attachment 616928 [details] [diff] [review]
patch v3

I don't think we can enable this by default without some clever heuristics preventing state or data loss (sound playing, video paused, plugin active, text entered where it wouldn't be restored?).
Attachment #616928 - Flags: feedback-
We don't need to enable it by default. I think it's a benefit for people with lots of tabs. Most likely not for the casual user.
To begin with, we should have an option to manually unload a tab. This process can then be automated with some heuristics.
(In reply to Ruben Petersen from comment #36)
> I am most interested in this not for reasons of RAM use but browser
> usability ("snappiness"). I want a browser that is snappy regardless of how
> many tabs are in the session. Unfortunately, for example with Bar Tab Lite
> currently, after a few hundred tabs (say 300-500) Firefox becomes very very
> slow and is prone to freezing up for, say, 2-10 seconds at a time. Mundane
> tasks like copying the URL of the current page into the clipboard take
> forever. Issues with regard to big session sizes need to be tackled within
> Firefox, not extensions.

I'm inclined to think the issue you describe is not caused by using too much memory. 


In reply to Tim Taubert [:ttaubert] from comment #67)
> We don't need to enable it by default. I think it's a benefit for people
> with lots of tabs. Most likely not for the casual user.

not only that, unloading of tabs can have a detrimental effect. one of the things I HATE about chrome, is what it does to inactive tabs.

I'd be pretty leery of any automatic approach
(In reply to Siddhartha Dugar [:sdrocking] from comment #68)
> To begin with, we should have an option to manually unload a tab. This
> process can then be automated with some heuristics.

At bare minimum,that's where I would start:tab loading on demand is IMHO as of now crippled by the inability of unloading tabs (either automatically or on demand),given that on demand loading in all likeliness fits heavy tab users,the scenario is that once you've cycled through a significant amount of them you'll end up with the same (very) high memory usage and lack of responsiveness that you were trying to avoid in the first place,only delayed.
The weird thing IMHO is that on one hand Firefox encourages the use of multiple tabs (tabbed browsing,then the addition of Panorama that pushed the concept further),on the other hand still doesn't provide to date all the tools to satisfactory deal with this,like the tab unloading feature we're discussing here or (different bug,I reckon,that's just to clarify my point) the full integration of Panorama across different windows and with the bookmarks library.
With regard to data loss, there are two simple things you could do
a) set up whitelisting of sites/tabs you never want unloaded
b) enforce long enough minimum timeout (1 hour?) to ensure eshop sessions have timed out to avoid double charge charges.. 

More graceful suspend/resume without enforced reload would be nice but perhaps bit more complicated to implement.
Guys, really the whole point of this bug is that every 2 months a new clone of BarTab pops up on AMO and none of them work. So why don't you just create an API which defines a method to unload the tab, and leave all this esoteria (i.e. data loss, how to do the GUI, etc) up to the plugin developer? Let's face it, the functionality of unloading tabs can only be done in-house, but you guys will never be able to figure out one perfect way to do it which will not **** off 30% of all Firefox users. So my suggestion is: do it, and leave it as an option to those who know. See what happens, how the plugins are used and how they evolve, and in time consider integrating a working solution into Firefox.
(In reply to cheater from comment #72)
> and none of [the BarTab clones] work.
Well that's just false. I'm running TooManyTabs and it's saved me loads of memory, made Fx much more responsive, and effectively replaced my bookmarks to boot. It doesn't automatically unload pages, but I'd hardly call that "not working".

> So why don't you just create
> an API which defines a method to unload the tab, and leave all this esoteria
> (i.e. data loss, how to do the GUI, etc) up to the plugin developer?
I like this idea.

> Let's face it, the functionality of unloading tabs can only be done in-house,
D'aww, you're cute.

> but you guys will never be able to figure out one perfect way to do it which
> will not **** off 30% of all Firefox users.
That is a risk I for one am willing to take. Perfection was never on the table, and I'd wager >= 30% of users are already **** off -- by how sluggish Fx gets with several tabs open.

> So my suggestion is: do it, and
> leave it as an option to those who know. See what happens, how the plugins
> are used and how they evolve, and in time consider integrating a working
> solution into Firefox.
Again, I like this, although I do have one reservation now that I think of it: plugins aren't expected to be compatible with (all of) each other, but they /are/ expected to be compatible with vanilla Fx. So, the sooner automatic tab unloading comes to vanilla Fx, the sooner it will be available to most users without potentially breaking something else.
@cheater
I think your idea is good, the UI is certainly seperate from the core plumbing. However, if there is no UI, then the only sensible thing to do is ship it preffed off. (let add-ons that expose a UI turn it on) Which creates the potential that it wont receive adequate testing. Givin the large demand you cite however, and adequate tests (we hope), perhaps this is OK in the short term.
(In reply to z1r0un from comment #73)
> (In reply to cheater from comment #72)
> > and none of [the BarTab clones] work.
> Well that's just false. I'm running TooManyTabs and it's saved me loads of
> memory, made Fx much more responsive, and effectively replaced my bookmarks
> to boot. It doesn't automatically unload pages, but I'd hardly call that
> "not working".

TooManyTabs is not a BarTab clone. It works differently and existed before BarTab. Get your facts straight before you try to be a smart-ass. Given that this is your first comment ever on the mozilla bug tracker (at least from what google can find) I'd say this wasn't unexpected. In fact, the only traces of any contribution to anything is you adding manga pictures to your favourites on deviantArt. I suggest focusing on that.

> > but you guys will never be able to figure out one perfect way to do it which
> > will not **** off 30% of all Firefox users.
> That is a risk I for one am willing to take. Perfection was never on the
> table

Judging by your lack of activity on this bug tracker and your email address being @gmail.com I am making an educated guess you do not speak for the Mozilla team and just yap here because you want to sound smart and important. The statements you made are left to be decided upon by the actual members of the Mozilla team. Please do not spam this thread.

> Again, I like this, although

Sorry to cut you short but this thread isn't a discussion between mozilla users, it's a discussion between mozilla developers with occasional input from a user towards the devs. Chatter between users is spam.

(In reply to scientes from comment #74)
> @cheater
> I think your idea is good, the UI is certainly seperate from the core
> plumbing. However, if there is no UI, then the only sensible thing to do is
> ship it preffed off. (let add-ons that expose a UI turn it on) Which creates
> the potential that it wont receive adequate testing. Givin the large demand
> you cite however, and adequate tests (we hope), perhaps this is OK in the
> short term.

In the meantime, this P2 bug has been stalled for nearly a year now, because since at least August 2011 people can't figure out how to properly do the user interface, mitigate data loss, and other UX nightmares which completely disappear if the functionality is opt-in through an addon.

@ttaubert: please consider releasing this as an API and give it serious thought. Other than lack of resources, I can't see any reason not to. Can you?
Let's stop this arguing right now, please.  It's clear that some people want this badly, and that others worry whether it can be implemented safely.  At this point, anything short of a patch is unlikely to bring this to fruition.
(In reply to cheater from comment #75)

*sigh* You know what? you're right. I'm sorry for being rude, and I'll leave until (if) I've contributed to the project.
(In reply to Nicholas Nethercote [:njn] from comment #76)
> Let's stop this arguing right now, please.  It's clear that some people want
> this badly, and that others worry whether it can be implemented safely.  At
> this point, anything short of a patch is unlikely to bring this to fruition.

As I understand, Attachment #616928 [details] [diff] is pretty much golden fwiw? Of course it contains no API. How does the team feel about putting this in the mainline, without the actual functionality turned on, but with an API which plugins can address?

(In reply to z1r0un from comment #77)
> I'm sorry for being rude
That's OK, sh!t happens.
> As I understand, Attachment #616928 [details] [diff] [diff] is pretty much golden
> fwiw?

Without r+ it can't land, and the f- is not encouraging.
(In reply to cheater from comment #75)
I don't think your reaction was really appropriate. He didn't say anything wrong. And when your only argument was that he is new to bugzilla, it seems you are too. Can you pick up the existing patch ?

(In reply to z1r0un from comment #77)
> *sigh* You know what? you're right. I'm sorry for being rude, and I'll leave
> until (if) I've contributed to the project.
Cheater was rude, not you.
For all those claiming that none of the bartab clones work, UnloadTab does work. In fact, it works bloody well. I have been using it since v8 through virtually all possible versions of FF, up to and including v14 aurora and now the just released v14 beta. UnloadTab v0.22 is no longer available but you can find it on one of the links below:

https://dl.dropbox.com/u/884148/unloadtab%40firefox.ext.xpi
http://mac.softpedia.com/get/Internet-Utilities/Firefox-Extensions/UnloadTab.shtml

Since it has been working fin through all versions, maybe the devs might want to consider its code? Or will I now be smacked too for bringing it up or for not having a @mozilla.com email address? (pun)
p.s. ... or maybe for misstyping @mozilla.org :)
Comment on attachment 616928 [details] [diff] [review]
patch v3

Review of attachment 616928 [details] [diff] [review]:
-----------------------------------------------------------------

A couple thoughts.
1. I agree with Dão in comment #66
2. We definitely shouldn't land anything on by default (I think you agree too)
3. I think we should have a minimum tab threshold (overridable of course) so that we don't hibernate with some small number of tabs open.

I'm obviously failing at reviewing this as is for a number of reasons, but one of them is that I'm hesitant to do any of this too automatically. So here's what I propose: how about we start by simply exposing the hibernate functionality, maybe in 2 steps (1. API, 2. UI) so that we can keep that under our control. What do you think?
Attachment #616928 - Flags: review?(paul)
API would allow people to experiment with it in extensions.  UI should probably have a blocklist of domains and/or URLs not to hibernate (perhaps with a few well-known examples - gmail?)  Pinned tabs shouldn't, etc.
(In reply to Randell Jesup [:jesup] from comment #84)
> UI should
> probably have a blocklist of domains and/or URLs not to hibernate (perhaps
> with a few well-known examples - gmail?)

Providing users with a foot gun and then requiring them to manually mitigate the damage seems like the wrong approach to me. A hardcoded blacklist doesn't sound sane either. If the proposed mechanism is incompatible with the web, then we need to go back to the drawing board.
(In reply to Randell Jesup [:jesup] from comment #84)
> API would allow people to experiment with it in extensions.  UI should
> probably have a blocklist of domains and/or URLs not to hibernate (perhaps
> with a few well-known examples - gmail?)  Pinned tabs shouldn't, etc.

Could it be as simple as: 1, provide the feature pref’d-off by default 2, exclude app/pinned tabs. Then user feedback on how and what breaks would be more valuable than assuming a blacklist is needed.

Does anyone have time to look at the attachment for review? so if it needs an update I could do that.
(In reply to Bogdan from comment #81)
> For all those claiming that none of the bartab clones work, UnloadTab does
> work. In fact, it works bloody well.

That depends on what you mean by "work".

It certainly provides some of the features formerly provided by BarTab, but from what I can see it doesn't seem to provide all of them. (The fact that it doesn't seem to be hosted by AMO also leaves me a bit leery of using it.)


As best I'm aware, BarTab itself did four things.

1: Prevent tabs from loading all at once on session restore, with an option to load them in the background a few at a time. This feature is now in core Firefox; it's less configurable than what BarTab provided AFAIK, but some of that configurability seems to be provided by the UnloadTab add-on.

2: Allow unloading of already-loaded tabs, either manually or on an "idle timer". This feature is reportedly implemented by UnloadTab, and is what this bug is about.

3: Provide a visual indication of which tabs are and are not loaded, by graying out (reducing the opacity of) the ones which are not. Screenshots seem to indicate that this may also be implemented by UnloadTab.

4: Provide the option to select the nearest already-loaded tab - if one exists in the current scope - when the current tab is unloaded or closed, rather than blindly selecting an adjacent tab (and loading it if it was unloaded). As far as I am aware, nothing other than BarTab has ever implemented this.

For my use case, all four of these features are essential; since UnloadTab doesn't provide the fourth one, it is not sufficient for me. I have therefore spent the past month or so hacking on BarTab to make the fourth one work again, and also make it work correctly with "tab group" scope instead of "window" scope (which it never did before AFAIK).

I would *love* to see these features make it into core Firefox, but only if they *all* do; as long as any one of them is still missing from the core, I will still need an add-on to supply it.
How about suspending tabs only by removing them from memory, e. g. save everything you know about the state (forms …) to disk and wipe out the compartment (someone proposed this in this bug before) and automatically restore it when someone accesses the tab (this will flicker, but it should not be too bad) – is this possible? But don't suspend if there is a plugin (hm, what about Flash Ads? – looks like we can't detect them) or video/audio (or workers? server event listeners? webrtc?) running or if it's an app tab. The TTL should be more than 1 hour on Desktop. Don't white/blacklist (the lists will never be accurate and thus annoy users). 

This will result in far less improvement but may help users a bit with long-running sessions. I think this is a feature not worth the time you put in. Make it simple and clean.
(In reply to Ruben Petersen from comment #38)
> Suggestion A:
> A way to unload a tab that saves it to disk with all entered form info etc,
> and that when restoring does NOT ask the website for an updated version or
> perform anything else to reload it (that in UX terms feels exactly like
> tabbing - so none of the "quick refresh" you see when clicking a tab BarTab
> Lite has so far prevented from loading).
> Suspend tab(s), write to disk, unload. Re-load on demand, resume.

I'm concerned about the privacy implications of this idea.  You'd be dumping random user input to disk; which could result in sensitive data being saved in an unprotected location without the users knowledge.
Form data is already stored since ~ Firefox 2 (and more with sessionmanager and similar extensions).

The issue is mostly one of a) plugin state, b) dynamic DOM state, c) other dynamic elements like MediaStreams, workers, PeerConnections, etc.  Certainly any page not allowed to go into bfcache will likely not be restored completely.  Since the proposal above is effectively little more than what would happen to the tab if we crashed/exited and restored, it's safe if non-optimal for many sites/tabs.

I'd suggest *if* this is done, there be an easy way for a page to opt out of this via simple content that can be checked before hibernating a tab (and probably via an HTTP header).  And then, given that, if someone wants to write a blacklist extension or one to expose the don't-hibernate state to the user it would be easy.
(In reply to Daniel Neely from comment #90)

> I'm concerned about the privacy implications of this idea.  You'd be dumping
> random user input to disk; which could result in sensitive data being saved
> in an unprotected location without the users knowledge.

Firefox already does this, for the session restore feature.
There was some discussion about this topic on a memshrink post:
https://blog.mozilla.org/nnethercote/2012/11/14/memshrink-progress-week-73-74/comment-page-1/#comment-8234

I think the best thought from the discussion amounts to: Why don't we exclude tabs from unloading that contain non-restorable content (*active* plugins, HTTPS, pinned tabs, etc)? 
The number of active plugins should be falling with click to activate coming.
I suppose this begs the question, are we able to label tabs with potentially non-restorable data with a high degree of confidence?
I confess to not having read the 90+ previous comments, so sorry if I'm treading old ground, but thought it was worth sharing.
About the privacy concerns: Let's encrypt the files and keep the key in memory (the key has to be added to sessionrestore, though).
If the tab can be put to sleep, so that no timers/events will run and no GC/CC runs over the objects, then the memory used can be sent to swap by the OS and will mostly be out of the way.
Blocks: 816381
Assignee: ttaubert → nobody
Hi,

are there any updates to this?

Thanks
It's nice to have add-ons that support related features. From the comment #99, dormancy stops support for Firefox 25+. Others all have severe performance degrading issues on a low profile machine running Xubuntu 12.04. Not sure if there's some correlated problems on the API they use?
See Also: → 865594
Whiteboard: [Snappy:P2] → [Snappy:P2] p=0
Hello there!

Some weeks ago I have filled a bug report for the huge memory Firefox can use (in my usage it got to 2.2 - 2.4 GB, Facebook alone was consuming 1+ GB of memory (lots of chats + normal usage)): https://bugzilla.mozilla.org/show_bug.cgi?id=962943
I got kind of no where, but I was sent to this thread, which was very good; some of my offered solutions are here, some are not (like limiting the memory amount, free old objects, etc.).

Meanwhile I have tested two of the plugins offered in #c99: bartab-heavy and unloadtab. They seem to have quite what I want, but.. they do not release memory. They unload the tab, but the memory is not freed of that tab's content. So they are quite useless. Maybe Firefox does not allow tabs memory to be released from addons?

The way I would like it to work is just like Firefox starts with many tabs from previous session: only pinned tabs are loaded, plus the active tab. Anything else is just loaded on tab activation. I can start Firefox with 10-20 tabs but only use 300-500 MB. In my oppinion, this is how it should work.
(In reply to florin arjocu from comment #101)
> Hello there!
> 
> The way I would like it to work is just like Firefox starts with many tabs
> from previous session: only pinned tabs are loaded, plus the active tab.
> Anything else is just loaded on tab activation. I can start Firefox with
> 10-20 tabs but only use 300-500 MB. In my oppinion, this is how it should
> work.

https://support.mozilla.org/en-US/kb/tab-preferences-and-settings
"Don't load tabs until selected"
@Elbart - this works and it is great. I want the same thing for unloaded tabs by addons, to release memory while using Firefox :)
Speaking as the "last developer" cited for the "bartabheavy" add-on listed in comment 99:

The way that add-on unloads a tab is roughly as follows:

* Open a new, blank tab.

* Prevent Firefox from treating the new tab as "loaded".

* Copy the "restore saved tab" information from the tab being unloaded to the new tab.

* Move the new tab to be adjacent to the tab being unloaded.

* Close the tab being unloaded, in such a way that it does not get retained in the "closed tabs" history list.

This means that, from the perspective of most Firefox internal code, the "newly unloaded" tab is in fact a new tab.


Normally, closing a tab does not also automatically drop (or flush to disk, or what-have-you) all information associated with the tab. This is presumably in part because the tab may be reopened via the "recently closed tabs" history, and in part because it is/was just easier to code that way.

"BarTab Heavy" (which is not the name I gave it - it's just a mostly-fixed version of the original BarTab, updated to work with then-current Firefox and with Panorama) goes some distance out of its way to make sure the closed "tab being unloaded" does not get put on the history list, but so far as I recall, it does not take any special steps to clear the unloaded tab out of active memory. It seems unlikely that any of the other "unload tabs" implementations do either.

In order for something that works the way BarTab and its descendants do to let memory be properly released after unloading a tab, there would need to be some way for them to tell Firefox to drop/flush all information about that tab, *except* (in the case of some of the descendants) the session-store information which is used to restore not-yet-loaded tabs. As far as I'm aware, there is no such capability.


If Firefox were changed to more aggressively garbage-collect or flush to disk or otherwise clean up in-memory information about "recently closed tabs", that would probably provide at least some of the on-tab-unload benefit automatically, with possible beneficial side effects in other areas. That's probably not nearly as simple as it may sound, however, and it might be hard to guarantee that it wouldn't have undesirable side effects as well.


(Also: why is this bug in ASSIGNED status, when it's marked as assigned to Nobody? If it used to be assigned to someone, who then lost interest and abandoned it, shouldn't the status have been changed at the same time?)
Status: ASSIGNED → NEW
(In reply to Andrew Buehler from comment #104)
> If Firefox were changed to more aggressively garbage-collect or flush to
> disk or otherwise clean up in-memory information about "recently closed
> tabs", that would probably provide at least some of the on-tab-unload
> benefit automatically, with possible beneficial side effects in other areas.
> That's probably not nearly as simple as it may sound, however, and it might
> be hard to guarantee that it wouldn't have undesirable side effects as well.

The catch here is that this functionality of NOT releasing all information about a closed tab was added intentionally in some previous version of Firefox and was not there before. It is to speed up loading a page when clicking Back, or when restoring a closed tab. The amount of memory reserved (or number of tabs kept) for this is determined via the pref browser.sessionhistory.max_total_viewers . You may look up what value to put there for experimenting. In the default state, the number is inferred from the amount of total system RAM you have.
Great responses, things got a lot clearer.

@Andrew Buehler - the addon works, only the memory usage is not done as expected by Firefox part.

@aceman
So this is one of the things that make the browser consume lots of memory: remembering data for many back states. In the case of Facebook, just imagine how many states it remembers, as I can get back to any other place I was there;  people are very active on Fb, we click hundreds of pictures, links, pages, profiles, about pages, etc. 
I do not know if this is the only thing that makes a Facebook tab to easily get to 1 GB of RAM, but it could contribute to it. I would guess no, as chatting does not create permalinks to get this effect, so there are more things to handle there, maybe also talk to some Fb developers? :D


I could see that having 4GB of RAM sets max_total_viewers to -1. I could guess it means unlimited. But the system memory is not unlimited, when the browser gets to high memory usage, the OS creates swap. And then the whole computer gets slowly. In the end I must close Firefox or restart system to free it again. Not nice.

Until some deeper solutions, some faster ones could also include:
- allowing max_total_viewers also per tab. Having only a total max_total_viewers might not function for people with many tabs, but I would definitely want to have 1-2 for tabs with let's say, a list of links where I go back and forward between the list and the linked pages.
- limiting max_total_viewers per tab. I definitely do not need Facebook to remember 100 old states. How many people go back so much? For just a hand of people, the others do not need it so agressive.
- really allow API/addons to flush memory for tabs. It is my decision if I install an addon, but if I need that, why not allow it? I expect many happy people because of this and, who knows, it might get in the Firefox code if people like this interaction.
Firefox's current ability to restart with all tabs and their histories present, but without actually loading content until clicked, seems to keep those non-loaded tabs from consuming any real memory and pretty much does what we'd need in a feature like this. Could that routine possibly be adapted to unload tabs selectively without restarting Firefox?
Unfortunately, the reason the "don't load tabs until selected" functionality is so effective at preventing the tabs from taking up memory is because those tabs really have never been loaded in that browser session. The current "unloading" process which frees up memory there actually takes place when the browser is closed, and AFAIK is more or less inherently tied to that shutdown process.

Once a tab has been loaded, to unload it in a way which approximates that "never loaded" state, something would have to be done to purge from RAM the cached-in-memory information related to that tab - except for the parts necessary to reload the tab. That happens automatically when the browser is being closed, since everything is being cleared from RAM at that point anyway.

Doing it as a separate, non-shutdown-related step might not be impossible, but I wouldn't expect it to be much like anything the existing "don't load tabs until selected" functionality does, or probably much like what happens during the normal shutdown process either; offhand I don't know of any other already-implemented functionality that is much like it.

That's actually what I pointed out would be necessary in comment 104.


max_total_viewers doesn't really work well as a workaround for this, because you may well want to e.g. retain cached information for a deep history stack for (at least some) open tabs, but drop cached information for all (or all but a very few) closed tabs. I haven't been able to come up with an alternative model that would let us avoid that problem without being overly clunky, though.
Yes, exactly like #107 (#c107) suggests would be great. Plus some options, to allow some preferred tabs (domains) not to unload (maybe I do not want Gmail to unload, I like how it changes color when I get an email, so I am always up to date) + some timing options (unload non-active tabs after x minutes).

I am also thinking about Firefox learning my habits and take them into consideration. Imagine it can make some statistics that I really use Youtube and it makes it faster to "recover", so it caches more than in asdfgh.com, one site I rarely use.
I don't mean to sound difficult, but from reading though this thread, it sounds like the problem is that Firefox never actually unloads anything and doesn't have the ability to. In most software you'd call that a memory leak and it would be a major bug. Even though I understand that the history's cache and the "reopen closed" feature requires retaining data about content that isn't necessarily in use, there ought to be some limit -- either on which types of data are retained, or on the number of closed tabs/history states to remember, or some combination of those), and a method to enforce it. If there isn't a bug posted for that yet perhaps there ought to be.
(In reply to florin arjocu from comment #109)
> Yes, exactly like #107 (#c107) suggests would be great. Plus some options,
> to allow some preferred tabs (domains) not to unload (maybe I do not want
> Gmail to unload, I like how it changes color when I get an email, so I am
> always up to date) + some timing options (unload non-active tabs after x
> minutes).

Much (possibly all) of the functionality you're talking about, except for the full "purge cached data from RAM" part, was present in the original BarTab on which the built-in "don't load tabs until selected" functionality was based.

I filed a wishlist bug a while back asking for the essential parts of this missing functionality to be added to Firefox itself, since IMO the bare "don't load tabs until selected" feature is borderline useless without some of that additional functionality. The bug was WONTFIXed within half an hour with a dismissive comment.

I've got most of those features working in modern Firefox with a modified (and mostly not yet published) version of BarTab Lite, but of course unloading tabs that way doesn't release all the memory used by the unloaded tabs, for the reasons I've already described. I hope to get that add-on finished, polished, and released at some point, but with so many projects on my plate it's not clear when it will happen.

In the meantime, some people seem to have reported success using "BarTab Heavy", which is linked to in comment 99; it's a version of the original BarTab I ported forward to post-4.0 Firefox. It's not perfect (I believe there are still bugs in regard to how tab history is handled when unloading and then restoring a tab), and like everything else, it doesn't free up all the memory used by the unloaded tabs - but if it works for you, go for it.


(In reply to equazcion from comment #110)
> I don't mean to sound difficult, but from reading though this thread, it
> sounds like the problem is that Firefox never actually unloads anything and
> doesn't have the ability to.

It's not that it never unloads things (i.e. purges previously cached data from RAM); it's that it doesn't do it immediately, there's no apparent way to manually tell it to do it, and that it retains more than seems reasonable in some situations. (However, changing the defaults to keep only what would be reasonable in those situations would likely end up keeping too little in other situations.)

> In most software you'd call that a memory leak
> and it would be a major bug. Even though I understand that the history's
> cache and the "reopen closed" feature requires retaining data about content
> that isn't necessarily in use, there ought to be some limit -- either on
> which types of data are retained, or on the number of closed tabs/history
> states to remember, or some combination of those), and a method to enforce
> it. If there isn't a bug posted for that yet perhaps there ought to be.

As I understand it, there is a limit on the number of history states to retain; that's controlled by the max_total_viewers pref, discussed above. Unfortunately, it doesn't cover only closed tabs; it covers all history, apparently with one slot taken up per page in the history, and the figure appears to be global across all tabs.

I agree that this should work and perform better than it does, but I haven't been able to come up with a model that seems sane.
I am using these addons (UnloadTab seemed to be less buggy than Bar Tab Heavy) but I also watch the memory usage of my Firefox. The conclusion is that using the plugins makes no huge difference than no plugin at all. So they do not really clear my RAM memory, but at least it can help not adding more from AJAX / JS / flash ads / whatever the site does from time to time. At least in my experience.

> Much (possibly all) of the functionality you're talking about, except for
> the full "purge cached data from RAM" part, was present in the original
> BarTab on which the built-in "don't load tabs until selected" functionality
> was based.
>
Thanks for your responses, Andrew. I'd just like to note for anyone interested, until the functionality is available, my plugin of choice is one that simply makes a "restart" button available. It is of course a rather blunt instrument, but it makes all tabs and their histories available but completely unloaded, and ready to be reactivated individually with just a click. For my purposes this is a pretty effective "hack", even though I do hope a more elegant solution materializes.
There is "the great suspender" extension for chrome that actually works. I do not know if that's more elegant solution but there it is. I guess the fact chrome actually opens tabs in separate processes helps things along. 

Without suspender my chrome would grow to allocate >3GB, with it it howers around 1.5GB. Not perfect but big improvement and no need to get 16GB RAM.
@Olli is correct. Chrome's separate processes do indeed make it easier to suspend a tab.  In FF it's not that simple due to its current architecture.

Those reading this bug report thread may wish to read up on Nuwa, as I imagine it is the technology which will eventually solve this issue.
https://wiki.mozilla.org/NuwaTemplateProcess
http://blog.mozilla.org/nnethercote/2014/02/11/nuwa-has-landed/
No longer blocks: fxdesktopbacklog
Flags: firefox-backlog+
Whiteboard: [Snappy:P2] p=0 → [Snappy:P2] p=13
(In reply to Shaun from comment #115)
> Those reading this bug report thread may wish to read up on Nuwa, as I
> imagine it is the technology which will eventually solve this issue.

Nope. You're looking for electrolysis/e10s.
Points: --- → 13
Flags: qe-verify?
Whiteboard: [Snappy:P2] p=13 → [Snappy:P2]
Until Firefox developers will implement the "automatic" RAM release of unused tabs, centuries will pass.

Meanwhile, I would like to suggest a simple manual solution (also see bug 112361):

Firefox should include a possible user action, per tab ,to right-click it and call a command "Close tab and save its state".

Actually this feature is already implemented (whenever Firefox crashes, you can access each single tab at the after-crash page), however it is activated by the crash, and not by a user command.
Isn't that what Recently Closed Tabs and Recently Closed Windows is for? You can customize the maximum amount of tabs and windows that are saved by modifying browser.sessionstore.max_tabs_undo and browser.sessionstore.max_windows_undo in about:config.
(In reply to t8av from comment #119)
> Meanwhile, I would like to suggest a simple manual solution (also see bug
> 112361):

The referenced bug is completely unrelated to what you describe.

(In reply to Emanuel Hoogeveen [:ehoogeveen] from comment #120)
> Isn't that what Recently Closed Tabs and Recently Closed Windows is for?

IMO yes.

Sebastian
(In reply to t8av from comment #122)
> 'Recently closed tabs' is exactly what I meant, but the way it is currently
> implemented is not usable

Firefox stores the browsing history in your Firefox profile directory, which can be controlled via Options > Privacy > History. There is a point 'Show All History' at the bottom of the history list, which opens a dialog, where you can search within the whole browsing history.

> Since Firefox is constantly crashing, this feature of 'Recently closed tabs' is highly important
If Firefox is constantly crashing for you, you should send crash reports to Mozilla.

(In reply to t8av from comment #123)
> #2 - The user can save any tab that he/she wishes (right-click on the tab
> and select 'save this tab'). These saved tabs will never be deleted, unless
> the user decides otherwise.

This option already exists. It's called 'Pin Tab'.

As this discussion is unrelated to this bug report and Bugzilla is not a forum, I advise you to first ask at https://support.mozilla.org/ before you create new bug reports for your suggestions.

Sebastian
Sebastian,

1. 'Browsing history' is not related to my suggestion. I was talking about saving 'tab state' to hard disk, which is something entirely different.

2. 'Pin tab' is also not related to my suggestion, since it does not save anything to HD. I was I was talking about saving tab to HD in order to release RAM and prevent crashes.

3. I already spent many hours on Firefox crash reports (including installing nightly), is there any point to spend more time on this, when the solution is elsewhere?

4. To clarify myself further - What I want is exactly the after crash page, but accisible also when there is no crash - you can access any tab that was opened in the past. Whenever Firefox becomes slow, I can of-course terminate the process of firefox, just to cause a crash, so I can access that page, but this is quite silly, isn't it?
(In reply to t8av from comment #125)
> Sebastian,
> 
> 1. 'Browsing history' is not related to my suggestion. I was talking about
> saving 'tab state' to hard disk, which is something entirely different.
> 
> 2. 'Pin tab' is also not related to my suggestion, since it does not save
> anything to HD. I was I was talking about saving tab to HD in order to
> release RAM and prevent crashes.
> 
> 3. I already spent many hours on Firefox crash reports (including installing
> nightly), is there any point to spend more time on this, when the solution
> is elsewhere?
> 
> 4. To clarify myself further - What I want is exactly the after crash page,
> but accisible also when there is no crash - you can access any tab that was
> opened in the past. Whenever Firefox becomes slow, I can of-course terminate
> the process of firefox, just to cause a crash, so I can access that page,
> but this is quite silly, isn't it?

So basically you're asking for an option to add a tab in an improved (and separate) version of the bookmarks, which saves the entire tab state (not only URL and page title) and closes it.
A feature like that would be much appreciated (I'd be the first to use it) and extensions like TooManyTabs are insufficient if you need to save a tab state. I also agree that it would be probably more feasible than the automatic unload object of this bug report and that it's related to it, but your proposal is about a manual unload, not an automatic one. I think you should open a new bug report and tell us the bug number, so the two features are kept separated and we can follow, discuss and vote your proposed feature there. Maybe this bug should depend on the new one, since the manual unload could be a step towards the automatic one.
1. Matteo, thanks for the support.

2. I already opened such a bug (bug 1123261), but someone marked it as a duplicate of bug 675539 (this page), that's why I continued the discussion here.

3. Do you think I should open a new bug with more details and explanations? (Maybe I should quote you: "an improved, and separate, version of the bookmarks, which saves the entire tab state").

4. If not - how to reopen bug 1123261 ?

5. As I already said. there is no real need to implement my suggestion, as it is already implemented in Firefox (this is exactly the after-crash page where you can open the tabs before the crash), but not accessible to the users in a normal mode of operation.

6. Regarding bug 675539 (the bug of this page): maybe let the user determine, in preferences, how long a tab is not used, until it is unloaded from RAM, and saved to Hard disk, so no need for a sophisticated automation mechanism.
I have reopened bug 1123261 and updated its title to distinguish it better from this bug.
See Also: → 1123261
(In reply to Nicholas Nethercote [:njn] from comment #128)
> I have reopened bug 1123261 and updated its title to distinguish it better
> from this bug.

You actually forgot to reopen it. I did that now and moved it to a more appropriate component.

Sebastian
(Now talking about bug 675539 - this page)

There are two different questions:

1. When to do the automatic unload of tabs (unload from RAM, and save to hard disk).
2. Which tabs to unload.

I would like to suggest the following possible cases/actions:

Normal operation
----------------
1. User definition based on expiry time - the user defines in his/her preferences how long from the time he stops using a tab, until this tab is unloaded. Not using a tab means no scrolling or data inputs in this tab.

2. At any given time, Firefox keeps only N loaded tabs (N=user defined). The rest will be unloaded. FF can choose the least-used tabs to be the ones that are unloaded. The decision which tabs to unload, can also be based on memory-consumption of each tab.

emergency cases
---------------
Emergency means that Firefox is going to crash, or extreme slowness.
In such cases, all tabs, except the active one will be unloaded, or even better - unload ALL tabs and after that reload only the active tab.

Triggers:
1. Memory usage of the browser - whenever the browser uses too much memory (maybe user-defined?).

2. Browser slowness.

3. Use request - if the user sees that the browser becomes very slow, he/she can make an  "emergency action".
@t8av - don't forget the exceptions. I would not want some tabs to be unloaded, right click-ing the tab and selecting "Don't unload this tab" could work just fine (check). Uncheck and it becomes unloadable.
(In reply to florin arjocu from comment #131)
> @t8av - don't forget the exceptions. I would not want some tabs to be
> unloaded, right click-ing the tab and selecting "Don't unload this tab"
> could work just fine (check). Uncheck and it becomes unloadable.

Exceptions only for normal operation. However, if the browser is going to crash, and unloading is what can save it, then I guess that no tab has immunity against unloading.
Open issue - 

When unloading a tab, does it mean that the tab will be also closed?

I suggest that it will depend on user configuration:

If the user sets 'Close unloaded tabs' - then when a tab is unloaded, it will also be closed. The user can always reopen & reload it from the 'tabs restore' page.

If the user sets 'Keep unloaded tabs open' - then when a tab is unloaded, it will be still open but with zero content (blank). The only content of the tab will be a link or button: 'Reload this tab'.
@t8av - yes, exception from unloading it during normal work.

Anyways, in my Ubuntu laptop, I did not see a Firefox crash in a long time (I forgot how it looks like) so I am only interested to see the normal unloading done, any other thingy is optional for me.
@t8av - no, do not close tabs on unloading, just clear the memory (if possible), just like it is loaded on browser restart (with Keep last session tabs activated). Until done, it might be a workaround to just copy some data from a tab to a new one and close the original one to free memory.
Florin, you are lucky, Here Firefox crashes few times per day. Did you try to scroll down on very long facebook/twitter image albums (which is called 'infinite scrolling')?

Regarding close-or-not for unloaded tabs, I think it is for the user to decide (use-configuration).
(In reply to florin arjocu from comment #131)
> @t8av - don't forget the exceptions. I would not want some tabs to be
> unloaded, right click-ing the tab and selecting "Don't unload this tab"
> could work just fine (check). Uncheck and it becomes unloadable.

I think that would rather clutter the UI, as that option would likely just be used by a handful of people. I'd rather improve the restoring mechanism, so that the tab will be loaded again exactly as it was before the unload.

(In reply to t8av from comment #132)
> However, if the browser is going to
> crash, and unloading is what can save it, then I guess that no tab has
> immunity against unloading.

You can't detect a crash before it happens.

(In reply to t8av from comment #133)
> Open issue - 
> 
> When unloading a tab, does it mean that the tab will be also closed?

That would be bad UX. People would wonder why their tabs are suddenly disappearing.

(In reply to t8av from comment #136)
> Florin, you are lucky, Here Firefox crashes few times per day.

Then you should report those crashes via the Mozilla Crash Reporter[1] and file bugs for them and/or create a new profile[2] or reset your current one[3].

Sebastian

[1] https://support.mozilla.org/kb/mozillacrashreporter
[2] https://support.mozilla.org/kb/profile-manager-create-and-remove-firefox-profiles
[3] https://support.mozilla.org/kb/refresh-firefox-reset-add-ons-and-settings
Daniel, you miss the point - whenever there are many tabs open, and Firefox crashes, it is a mess, so some users (including myself) prefer that most tabs will be safely (and orderly) saved to hard disk, so the user can reload any tab that is needed.

(To completely prevent Firefox from crashing, all you need to do is to run the following windows command: C:\Program Files\Google\Chrome\Application\chrome.exe).
@t8av, @Daniel: Please stop your discussion about Firefox crashes. This issue is not the right place for this.

Sebastian
This is really a subset of a slightly broader issue, that there isn't an easy API to allow extension developers to direct Firefox to withdraw or suspend specified resource-consuming services used by specified tabs.

If Firefox is running slower or more "resource hogging" than the user wishes, it doesn't matter if the system is low spec and the use modest, or the system high spec and the use intense. The same problems arise: the user wants a way to forcibly direct Firefox to reduce its resource use on specified tabs, perhaps going as far as to cache their state or kill almost all of their memory/CPU use until they are foregrounded or "activated" again. 

Memory is just one resource consumed by background tabs.  CPU time is also used to service DOM events, plugins, and some items with built-in timing (GIFs?). Chrome presumably consumes CPU time even on idle tabs - for example, events may iterate through tabs or respond to (or act on) background tabs, or "hooked" calls. So depending on how severely system resources are to be freed, the broader picture of freeing up resources on a background tab is quite granular.  It could involve one or more of: 

* decaching static or cachable objects (text in a textbox, images, flash objects, etc)
* caching (to disk or sessionstore) or deleting the DOM state
* losing the DOM state entirely (next tab access will treat tab as if reloaded?)
* JS engine suspending threads from the tab (functionality already exists via the debugger)
* JS engine not generating (most) events for that tab
* plugins created by that tab being send a message to pause, suspend or close
* other?

The question of which tabs are restricted this way isn't core. It's off-topic. If an API exists so that an extension author can set+get the resource suspension state for a tab, then it's down to extension authors to decide what criteria/choices are offered, and what resources are acted on, out of those provided by the API. For me the problem reduces to the absence of a generic API that extension authors can call on for the job, so extensions are lacking and the few that exist have to hack round the issue with unofficial calls and stop working after a year or so.

So from here, I think what's missing is to figure out the types of "resource suspension" which would be meaningful and not onerous to include in the core, then if anyone takes up the challenge, add an API to set and get resource suspension/resource active status for a tab. Even if the API starts out with very few types of resource saving, and most of the desired actions inoperative, it would still help and the actions it can handle can be added over time.

At its simplest, the needed API "getter" would take a tab (or tab list) and return a bitwise indicator showing which resources have been withdrawn for that tab; the "setter" would do the same but provide a bitwise indicator. The effect would be "if this tab is in the background of a window, Firefox will withdraw these resources from it". (Obviously we don't want the foreground to be forcibly frozen)

For example, the "do not service content  events" flag on a tab would mean that events were suppressed at source for that tab. The "do not execute content scripts" flag would presumably reuse the existing debugger "suspend" function.  

The absence of a generic API mechanism by which an extension can forcibly withdraw specified "system resources" such as memory/CPU from tab(s), feels like a better and more solid place to focus any work on Firefox Core, and gives this bug/feature a more robust context. 

What issues would such an API give rise to, which need resolving first?
(In reply to Stilez from comment #144)
> This is really a subset of a slightly broader issue, that there isn't an
> easy API to allow extension developers to direct Firefox to withdraw or
> suspend specified resource-consuming services used by specified tabs.

Very good idea! Though I'd say you should create a new report for that and mark it as blocking this one.

> Memory is just one resource consumed by background tabs.  CPU time is also
> used to service DOM events, plugins, and some items with built-in timing
> (GIFs?).

Right. I adjusted the summary to reflect that.

Sebastian
Summary: Automatically unload (stall/hibernate) longly unused tabs to free RAM → Automatically unload (stall/hibernate) longly unused tabs to free resources
I just got an idea: some of this unload action I might be in there already Firefox for Android, as on smartphone, one cannot afford to "eat" lots of memory without releasing it when it is not needed.
(In reply to Sebastian Zartner [:sebo] from comment #145)
> Very good idea! Though I'd say you should create a new report for that and
> mark it as blocking this one.

I totally agree with Stilez, so please do that. If you don't / can't mark it as a dependency for this bug report, at least write here the bug number for reference, so we can follow it. Thanks!

(In reply to florin arjocu from comment #146)
> I just got an idea: some of this unload action I might be in there already
> Firefox for Android, as on smartphone, one cannot afford to "eat" lots of
> memory without releasing it when it is not needed.

AFAIK Firefox for mobile uses a "lightened" version of the code (or different infrastructure: not sure, can't remember and don't have time to search right now), so if it already has a similar functionality it probably needs some rewriting before the porting, which is better than reinventing the wheel in any case.
(In reply to florin arjocu from comment #146)
> I just got an idea: some of this unload action I might be in there already
> Firefox for Android, as on smartphone, one cannot afford to "eat" lots of
> memory without releasing it when it is not needed.

This was added to Fennec in bug 803575. Doing it on desktop isn't necessarily the right thing to do. From https://blog.mozilla.org/nnethercote/2012/11/14/memshrink-progress-week-73-74/comment-page-1/#comment-8234:

"An obvious follow-up idea is to add tab expiration to desktop Firefox.  But the bug tracking that idea has seen more heated discussion.  The reason is that tab expiration might cause data loss in some cases.  This is less of a problem on Fennec because (as far as I can tell) there is less of an expectation that in-flight data will be saved on mobile devices.  For example, having processes killed due to memory constraints on mobile is much more common than on desktop.  Still, I expect plenty more arguing before this issue is resolved one way or the other."

Also, comment 76 still applies, and additional discussion without an attempt by someone to implement is unlikely to achieve anything.
Its important that everyone will understand how MSIE (Microsoft Internet Explorer) deals with crashes on 'infinite image scrolling' (facebook, twitter, etc') - what MSIE do is to UNLOAD PART OF THE TAB.

For example, if you scroll-down in a page were there are 20000 images, after a while, when many images are already loaded, the browser will crash. No browser can keep in RAM too many images at once. This is common sense 
 
What MSIE do is to UNLOAD the upper images of the page. Typically when you scroll down, you do not often scroll back up.

From what I have seen, when IE is loading too many images, it starts to release memory (unload upper images). This process creates a delay - The browser stops responding for a while, but it does not crash (unlike Firefox). After the delay, you can continue working normally. 

Since the first images of the page were unloaded, if you scroll back up, you have to wait until they are loaded again.

It seems to me that this is the best possible solution also for Firefox.

Since many people need this feature (while some are against), I think it should be at the users-preferences:

"Auto unload older tab multimedia" - TRUE/FALSE

This applies to images, and also to HTML5 videos that sometimes being embedded in large numbers in a page.

One last remark - this bug page seems to me like an never-ending-discussion. I suggest that to save time, you simply implement my suggestion, in a way that is also reversible (using user preferences), and just see if it does any good or not.
Depends on: 1128502
>Since many people need this feature (while some are against), I think it should be at the users-preferences

I would second that: why not add this as an option ,just like loading tabs on demand, and make it non active by default, so that folks who don't need this/have no idea of what this is about just leave it (hopefully) unchecked.
BarTab Heavy is the addon I've been using to do this, but it appears to be abandoned.  Hopefully this option gets built in before the addon signing requirement does, because we won't be able to use older / abandoned addons aftewards!
(Grr. Some bizarre combination of accidental keypresses managed to invoke Firebug and delete the textarea when I'd nearly finished writing my comment...)

To the best of my awareness, BarTab Heavy is based on work I did to update the original BarTab for Panorama (et cetera).

Rather than continue working on that, since I'd run into so many snags in various places, I forked BarTab Lite (or possibly someone else's fork of the same, I no longer recall offhand) and started trying to add the extra, "heavy" BarTab features back in to it, for private use. Some of them are still missing, but others - including, last time I checked, "unload tabs automatically after a delay" - are working just fine.

I have too many projects as it is, so since this is working fine for my purposes it's been on the back burner for a while, but I've always meant to release BarTab Restartless publicly at some point. If/when the addon signing requirement gets implemented as expected, ping me (since I may not notice right away) and I'll see what it would take for me to release it in its current apparently-functional state. (I'll need to verify that it still works with latest Firefox at that point, of course.)

That said, I expect that I'd recommend that anyone who cares about addons just run the allegedly upcoming "unbranded" Firefox build, which allegedly won't include that requirement - in much the same way as I recommend that anyone who cares about not hitting UI regressions run the ESR, simply so that they only have to check for such regressions before upgrading (and try to find out how to mitigate the ones they find) about once a year, rather than once every six weeks.
I have been trying to use addons for this for many months, but *ALL* of them either have crippling bugs (tabs suddenly don't accept keypresses anymore, tabs suddenly don't close properly anymore, ...) or they don't work with firefox nightly, which I use because the lagginess and hanging of the stable version is unbearable.

While I managed to use such an addon successfully on firefox stable for quite some time, it was still an unfavorable experience in terms of confusing options and such for a simple functionality that should simply work out of the box without being too obtrusive.

It would be therefore really nice if you could just finally add this to core behavior in a way that simply works(TM) with a sane default (like, unload old tabs after they haven't been interacted with since 50 newer tabs or something).
(In reply to t8av from comment #155)
As long as there's an option of course. :) 
I have 32 GB of RAM, so I just want Firefox to use as much RAM as possible to keep it snappy and fast, instead of having to load it from my drive first.
(In reply to Lily from comment #156)
> (In reply to t8av from comment #155)
> As long as there's an option of course. :) 
> I have 32 GB of RAM, so I just want Firefox to use as much RAM as possible
> to keep it snappy and fast, instead of having to load it from my drive first.

Great statement. I also have a nice statement of my own:

Since I am a billionaire, and to demonstrate my wealth, I purchased, last month, a supercomputer from China.

It has 200,000GB of RAM and it can perform 1.8 thousand trillion floating point operations per second (petaflop).

Please adjust Firefox to fit the performance of my PC.

Thanks.
For the best case it should be a couple of options, IMHO.
Some users want to free RAM but some have it enough and want just to suspend all CPU activity in old tabs (keeping them in RAM).
There seems to be no addon for the latter case for now. "Suspend background tabs" was (?) a solution, but it is discontinued.

Facebook might be an example--its chat function generates frequent network requests and might be a problem if your traffic is limited and you have a lot of FB tabs opened.
Depends on: 1284886
For anyone with the same issue, https://addons.mozilla.org/en-US/firefox/addon/tab-suspender-tab-unloader seems to provide the requested functionality.
(In reply to Kristian Klausen from comment #160)
> For anyone with the same issue,
> https://addons.mozilla.org/en-US/firefox/addon/tab-suspender-tab-unloader
> seems to provide the requested functionality.

That addon doesn't work with me. It locks up my browser many times and it makes my browser (firefox x64) take 15 minutes to be usable, since I start it, after taking almost 7GB of RAM.
Without it, it stays at much more stable ~900MB of RAM (I do use a lot of tabs opened ~200)
To <https://github.com/rNeomy/tab-suspender/issues>, please, for issues with Tab Suspender (Tab Unloader).
My intention too. I was just giving my feedback due to Kristian's comment.
I do not see any reason to unload the tabs by time, in addition to the negative effect of the memoryloss, it does not make much sense (unused ram = wasted ram). I would like to see the function of freeing up memory only when necessary, for example when the memory in the system runs out (because of the firefox demands or the user opened another application). In general, I would like to see something like that in the firefox https://developers.google.com/web/updates/2015/09/tab-discarding
To recap: There are several reasons to do this.

- on a system with a lot of memory pressure you might find yourself with long pauses due to an oom-like behavior
- memory freeling will lag behind new memory allocation requests. This means the system is designed to run out of memory, and to fail. I can easily see ff getting oom'd by the OS especially on slower systems
- operating systems run other things than a single browser process. Maybe the user wants one of those other things to use as much memory as is available - and not ff
- maybe you are running multiple processes of ff. Which one should be the one using the excess memory?
- last but actually most important: tabs not only use memory, but also cpu. This is very important and often a reason for terrible UX and stability issues with many tabs open.
To qualify the above a bit better: if you think not using tume-based unloadong will work for you, that is fine, and you should be allowed to do this. No one loses this way. But it's also very important to be able to do time-based as well.
Blocks: 1383979
I, for one, am just interested in the CPU usage part, see my related Stackoverflow question https://stackoverflow.com/questions/49315743/how-to-pause-firefox-of-invisible-gui-sessions .

I will test the "Auto Tab Discard" add-on for now (https://addons.mozilla.org/en-US/firefox/addon/auto-tab-discard/reviews/), which is the successor of Tab Suspender.
With Dormancy [1], I've been experimenting with patterns that achieve the balance of minimizing use of system resources while also keeping the recent+relevant tabs alive.

I started with multi-window usage, which is by far the minority of users [2], but is how I've been using the browser since tab groups went away.

Waiting 15 minutes before hibernation, combined with keeping tabs in the active window alive, achieved something that feels right to me. I'm getting better battery life, and the tabs I'm actively working with tend to stay alive and ready.

I'm going to switch back to single-window use of the browser and start testing more complex hibernation heuristics such as:

* Recency: How long ago was the tab opened? How long was the last-active session in the tab? (eg, was user just traversing tabs, or actually using the tab)

* Relevancy: Same domain? Is opener or parent of active tab?

* Adjacency: How close or far away in the tab strip is the candidate tab from tabs active in the last N minutes?

If you have suggestions for hibernation heuristics, add your suggestions at https://github.com/autonome/Dormancy/issues/20

I'd also like to develop some deterministic measurements of how well the heuristics are working. But that might not be realistic to do in an extension... eg, measure median time to reactivation for hibernated tabs, or when a tab is activated maybe we could measure how often the tab was alive vs hibernated. Measuring power consumption (cpu cycles?) and memory consumption would be good as well - but far more difficult to set up and measure in a verifiable way.


[1] https://addons.mozilla.org/en-US/firefox/addon/dormancy/

[2] https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2018-03-22&keys=__none__!__none__!__none__&max_channel_version=nightly%252F61&measure=SCALARS_BROWSER.ENGAGEMENT.MAX_CONCURRENT_WINDOW_COUNT&min_channel_version=nightly%252F58&processType=*&product=Firefox&sanitize=1&sort_keys=submissions&start_date=2018-03-12&table=0&trim=1&use_submission_date=0
Abstracting away from what your plugin does and returning to this bug report, it bears saying that if you have, say, 10 windows open, you're only ever going to use 1-2. Many people will have all windows maximized, or at half width (two windows, full-height, side by side on a single monitor). This means only 1-2 will be visible at a time. The others will be occluded. I think Firefox should have a way of requesting this sort of information from the window manager. Then it can tell the api whether a specific tab is in a window which is occluded or visible. I think this would be very valuable.
This was actually looked into for pausing of video decoding (not stopping playback) of windows which were fully occluded.

Turns out only MacOS has a proper API to tell about full window occlusion. (APIs/solutions on other platforms are insufficient.)
It's possible to figure this out on Linux using xrandr (to get desktop geometry) and xdotool (to get window geometry) and xprop/xwininfo (to find out what windows are assigned to our process - or keep track yourself). Is there a bug number for this?
While I'd love to see proper tab unloading, I think Firefox should start by just suspending threads/processes of non visible tabs. Chrome is going to do it in the near future. Not having 100% CPU used by javascript of invisible ads would already be a huuuge gain. The O/S would also have much more freedom to swap out these tabs resulting in more RAM usable by current tab and the rest of the OS. When reactivating the tab Firefox would just unpause the threads/process and there's no need to worry of losing page's state. This seems like a bit of a low hanging fruit to me. Even if you consider outright pausing bad, just throttling the threads would probably make a big difference. Again see what Chrome plans are in this regard (https://developers.google.com/web/updates/2017/03/background_tabs and https://docs.google.com/document/d/18_sX-KGRaHcV3xe5Xk_l6NNwXoxm-23IOepgMx4OlE4/edit#).
I think Mozilla should consider this very seriously. Just think of the carbon footprint of the wasted electricity world-wide, let alone people's productivity and user experience. Maybe acting before Google on this would make some people go back to FF from Chrome. I was surprised how fast Firefox feels on my Android tablet compared to my 6 core/12 thread 32GiB RAM workstation. I think the difference is that the mobile version unloads tabs. When I have 2 or 3 tabs open in my desktop FF everything is snappy, but I usually end up with dozens of tabs open after an hour or so of browsing and the whole of Firefox gets sluggish like hell.
I'm a developer and I cannot think of a reason why websites should do anything at all when not active other than finishing an already ongoing HTTP request and some special cases like spotify/soundcloud playing music or gmail notifying of new emails. Even the ad providers would prefer to pay only for ads that users actually see rather than displayed in a rendertexture just to burn some cycles. These should be an exception. Such pages could and should ask for an explicit permission to continue running in background. Maybe pinned tabs should be exempt from pausing/unloading as the user already expressed the intent of having these alive and ready all times. Thinking about my own usage I noticed that even if I open two or more tabs that are interactive or play media the fact that both start something in background is just annoying. I have to go to just opened tabs to pause any videos or mute them, just to return later to unpase. Youtube folks must've noticed it too and are pausing content if I open youtube links as background tabs (though annoyingly they start anyway after some time, like a few hours without me activitang them).
Does Mozilla have any data on tab usage? If so, it of course should be considered. Though in my not very humble opinion, even if 99% of users keep less than 5 tabs open, the remaining 1% is probably the more vocal part like myself that can swing the overall browser market share one way or another.
Anyway it would be nice to have an official stance on the matter from Mozilla and maybe a long term high level goals statement.
(In reply to Maciej Kacper Jagiełło from comment #172)
> While I'd love to see proper tab unloading, I think Firefox should start by
> just suspending threads/processes of non visible tabs. Chrome is going to do
> it in the near future. Not having 100% CPU used by javascript of invisible
> ads would already be a huuuge gain. The O/S would also have much more
> freedom to swap out these tabs resulting in more RAM usable by current tab
> and the rest of the OS. When reactivating the tab Firefox would just unpause
> the threads/process and there's no need to worry of losing page's state.
> This seems like a bit of a low hanging fruit to me. Even if you consider
> outright pausing bad, just throttling the threads would probably make a big
> difference. Again see what Chrome plans are in this regard
> (https://developers.google.com/web/updates/2017/03/background_tabs and
> https://docs.google.com/document/d/18_sX-KGRaHcV3xe5Xk_l6NNwXoxm-
> 23IOepgMx4OlE4/edit#).
> I think Mozilla should consider this very seriously. Just think of the
> carbon footprint of the wasted electricity world-wide, let alone people's
> productivity and user experience. Maybe acting before Google on this would
> make some people go back to FF from Chrome. I was surprised how fast Firefox
> feels on my Android tablet compared to my 6 core/12 thread 32GiB RAM
> workstation. I think the difference is that the mobile version unloads tabs.
> When I have 2 or 3 tabs open in my desktop FF everything is snappy, but I
> usually end up with dozens of tabs open after an hour or so of browsing and
> the whole of Firefox gets sluggish like hell.
> I'm a developer and I cannot think of a reason why websites should do
> anything at all when not active other than finishing an already ongoing HTTP
> request and some special cases like spotify/soundcloud playing music or
> gmail notifying of new emails. Even the ad providers would prefer to pay
> only for ads that users actually see rather than displayed in a
> rendertexture just to burn some cycles. These should be an exception. Such
> pages could and should ask for an explicit permission to continue running in
> background. Maybe pinned tabs should be exempt from pausing/unloading as the
> user already expressed the intent of having these alive and ready all times.
> Thinking about my own usage I noticed that even if I open two or more tabs
> that are interactive or play media the fact that both start something in
> background is just annoying. I have to go to just opened tabs to pause any
> videos or mute them, just to return later to unpase. Youtube folks must've
> noticed it too and are pausing content if I open youtube links as background
> tabs (though annoyingly they start anyway after some time, like a few hours
> without me activitang them).
> Does Mozilla have any data on tab usage? If so, it of course should be
> considered. Though in my not very humble opinion, even if 99% of users keep
> less than 5 tabs open, the remaining 1% is probably the more vocal part like
> myself that can swing the overall browser market share one way or another.
> Anyway it would be nice to have an official stance on the matter from
> Mozilla and maybe a long term high level goals statement.

Just to correct my self. Seems like Firefox already has a throttling mechanism similar to Chrome since version 58. I guess I need to test without noscript or unloading to see if there's a noticeable difference.
See Also: → 1517149
See Also: 1517149

I believe that 1524501 is not a duplicate of this bug.
This bug is not about an event to act when memory is low while the other one is.
There's an overlap on purpose with a small difference.
I suggest to either change the title of this bug to include that or keep the other open as something which would be an addition to this bug which is about discarding when memory is low.
What do you think?

Flags: needinfo?(dao+bmo)
Flags: needinfo?(dao+bmo)
Summary: Automatically unload (stall/hibernate) longly unused tabs to free resources → Automatically unload (discard/hibernate) longly unused tabs to free resources / when running out of memory

I was aware of this bug but I had opened a new one because there's a significant difference between the two: in this bug we want to reclaim memory by unloading tabs that have long been idle even though we don't need that memory right away. Bug 1524501 is about forcibly unloading tabs when we know we're low on memory and not doing so would lead to an OOM crash. The latter is always positive for the user as we're introducing a small annoyance (reloading the tabs) in exchange for not crashing; the former might be annoying for no immediate benefit if we unload tabs that the user will come back to soon. See this experiment by :dietrich for some discussion on the topic.

I will resume my work under this bug since the title has changed but we should also have a separate one for the original intent, i.e. unloading tabs that have been idle for a long time even if available memory is plentiful.

Component: Session Restore → Tabbed Browser

Forcible unloading tabs when running out of memory is very important to introduce and very welcome.

But I am not sure about unloading tabs that have been idle for a long time - maybe user should be given message about that with options to chose from: unload these tabs or not.

Just set a option to enable the "unload idle tabs" (disabled by default) and, below, a place for the user to define the time to trigger the unload

As a alternative or even in parallel, only start to unload after X loaded tabs (let me guess about 20 tabs, but that can also be a option)

Attached patch [PATCH] WIP (obsolete) — Splinter Review

I've cooked up a very rough WIP that implements this functionality. There's still a few issues to be sorted though and I would like to write tests for this. The biggest issue I encountered is that I seem to be able to run into a race if I unload the tabs while switching through them. I don't know if this is a problem with my patch or with the discardBrowser() implementation.

Assignee: nobody → gsvelto
Attachment #616928 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Attached patch [PATCH v2] WIP (obsolete) — Splinter Review

This unloads the tabs in the correct order, first tabs not playing audio and not pinned, then pinned tabs not playing audio and finally tabs playing audio if nothing else is available.

Attachment #9043596 - Attachment is obsolete: true

A pop up with this information will be very annoying for me and much other users and shouldn't definitely be activated automatically. A push notification could be an option. In my opinion pinned Tabs and media tabs shouldn't be affected by this else I wouldn't pin them up. I already need to "restart" my Youtube Playlist every XX Minutes, affected media tabs would only increase the cycle of "restarting" it. You should think about a "Whitelist" instead of pinned and media tabs.

But as a hard fact, I cant use Firefox without a discarding Addon, else i would run out of memory (16GB), up to twice per day. I cant imagine this as a goal from Mozilla.

Possible Settings:

[X] Activate automatic discard of inactive Tabs after [30] Minutes and after [10] Minutes if a battery is used.
[ ] Also discard pinned and audio playing tabs.
[X] Display a Push Notification after discarding tabs. ("XX Tab(s) are discarded cause of inactivity.")
[X] Do not load discarded Tabs at start of Firefox to improve the startup time of Firefox.

This decreases waste of Memory and Energy.

(Yes CPU/GPU? hungry sites can also waste Energy. I won't see the difference on my overclocked I7 but on a mobile Device...!)

Sorry @all I don't know the difference between unloaded, discarded and so on. Not leeching memory and staying ready on a click is important for a prosumer with much tabs).

This option should not affect tabs with filled forms. This could result in bad UX. (Write a long text - going for a break - all discarded/deleted)

This removes the need to go through the browser-to-tab mapping when discarding
a tab and simplifies the relevant code. Besides being renamed discardBrowser()
was also split so that one can check if a tab can be discarded prior to trying
it.

This adds a mechanism that discards tabs when the browser detects a low-memory
scenario. Tabs are discarded in LRU order prioritizing regular tabs over
pinned ones, pinned ones over tabs playing audio and all of the previous over
pinned tabs playing audio.

Attachment #9043604 - Attachment is obsolete: true

(In reply to Gabriele Svelto [:gsvelto] from comment #186)

Created attachment 9045221 [details]
Bug 675539 - Make tab discard functionality work on tab objects directly

This removes the need to go through the browser-to-tab mapping when discarding
a tab and simplifies the relevant code. Besides being renamed discardBrowser()
was also split so that one can check if a tab can be discarded prior to trying
it.

It's called discardBrowser deliberately, because it throws away the browser. The tab (the thing that sits in the tab bar) remains...

(In reply to Dão Gottwald [::dao] from comment #188)

It's called discardBrowser deliberately, because it throws away the browser. The tab (the thing that sits in the tab bar) remains...

Makes sense, I'll reverse the name change, though passing the tab instead of the browser object still looks like a reasonable choice unless I'm missing something.

Blocks: 1529556

(In reply to Gabriele Svelto [:gsvelto] from comment #187)

Created attachment 9045222 [details]
Bug 675539 - Unload tabs in low-memory scenarios

This adds a mechanism that discards tabs when the browser detects a low-memory
scenario. Tabs are discarded in LRU order prioritizing regular tabs over
pinned ones, pinned ones over tabs playing audio and all of the previous over
pinned tabs playing audio.

Could we reload the tab if it is selected? discardBrowser does nothing if the tab is selected. Ex: If the current tab contains a serious memory leak, it is probably better reloading it than waiting for the OOM killer to kill Firefox.

[1] https://searchfox.org/mozilla-central/rev/dc0adc07db3df9431a0876156f50c65d580010cb/browser/base/content/tabbrowser.js#2200

(In reply to Kristian Klausen from comment #190)

Could we reload the tab if it is selected? discardBrowser does nothing if the tab is selected. Ex: If the current tab contains a serious memory leak, it is probably better reloading it than waiting for the OOM killer to kill Firefox.

That is an interesting idea but a bit disruptive for the user. For the time being I haven't considered evaluating how much memory a tab is using but since it's possible to get an approximation we might want to add this down the line as a measure of last resort.

Pushed by gsvelto@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/60a25e3f7b50
Make tab discard functionality work on tab objects directly r=mconley
https://hg.mozilla.org/integration/autoland/rev/a836b30ac070
Unload tabs in low-memory scenarios r=mconley
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 67

Will that new behaviour require a mention in our release notes?

Flags: needinfo?(gsvelto)

I believe, although not required, it should be mentioned in the release notes.

(In reply to Pascal Chevrel:pascalc from comment #196)

Will that new behaviour require a mention in our release notes?

Note that at this time, this is preffed to run on Nightly only for now.

(In reply to Mike Conley (:mconley) (:⚙️) from comment #198)

(In reply to Pascal Chevrel:pascalc from comment #196)

Will that new behaviour require a mention in our release notes?

Note that at this time, this is preffed to run on Nightly only for now.

We have release notes for Nightly :)

See Also: → 1495355

Ideally I'd say yes because this should significantly reduce the chances of Firefox crashing because of an OOM condition. However we'll need at least a couple of weeks worth of crash ping data to confirm my hypothesis. Also this is Windows-only ATM because it's the only platform where we can detect a low-memory condition.

Flags: needinfo?(gsvelto)
Fission Milestone: --- → M1

@Gabriele it's possible to detect low-memory condition on Linux: https://github.com/torvalds/linux/commit/eb414681d5a07d28d2ff90dc05f69ec6b232ebd2

Is there is separate bugreport about implementing automatic unload unused tabs on Linux, or I should register one?

The unloading part of this bug is system agnostic, what needs to be implemented is low-memory detection and I don't think we have a bug for that so feel free to open one and CC me. Thanks.

Depends on: 1532955

Can I get a bit of detail on how a tab is determined as a candidate for offloading? How long does it need to sit unvisited/unviewed, etc.?

Flags: needinfo?(dao+bmo)

(In reply to tahn from comment #204)

Can I get a bit of detail on how a tab is determined as a candidate for offloading? How long does it need to sit unvisited/unviewed, etc.?

All background tabs are candidates for being unloaded when there is a memory pressure event, there is no minimum tab lifetime. The order is outlined in the patch description above (Comment 187).

Flags: needinfo?(dao+bmo)

Is there an easily available option, or will there be one for the user to configure the threshold for a memory pressure event?

It is a huge problem with modern browsers that if you work from home on a machine with lots of system memory with applications that need lots of RAM which you also use privately, that a modern browser will happily gobble up 8GB+ if running for a while, making it impossible to launch actual work without manually terminating it. I understand this might be desirable on some entertainment-only machines to optimize performance, but in general this seems really like something that should be configurable.

(Especially since most modern desktop systems, unlike e.g. Android, lack a cooperative "please give up some of your memory" event for processes that is based on something else than the process itself deciding/observing that system memory as a whole is already critically low)

(In reply to jonas from comment #206)

Is there an easily available option, or will there be one for the user to configure the threshold for a memory pressure event?

It is a huge problem with modern browsers that if you work from home on a machine with lots of system memory with applications that need lots of RAM which you also use privately, that a modern browser will happily gobble up 8GB+ if running for a while, making it impossible to launch actual work without manually terminating it. I understand this might be desirable on some entertainment-only machines to optimize performance, but in general this seems really like something that should be configurable.

The threshold is not configurable, but there's a number of extensions that allow you to unload tabs manually or based on usage, last time of visit, etc... You might want to try one of those. This is just meant to save Firefox when the alternative is crashing.

Thanks so much, I didn't know where to start with this comment thread!

(In reply to Kestrel from comment #205)

(In reply to tahn from comment #204)

Can I get a bit of detail on how a tab is determined as a candidate for offloading? How long does it need to sit unvisited/unviewed, etc.?

All background tabs are candidates for being unloaded when there is a memory pressure event, there is no minimum tab lifetime. The order is outlined in the patch description above (Comment 187).

but there's a number of extensions

Last time I checked (admittedly a while ago) none of them worked with quantum, also they tend to offer something else other than raw total memory usage as trigger.

Also, I am wondering: if you want to avoid crashing, are you also avoiding OOM kills on linux?

Because right now on all big linux desktops firefox has kinda awful behavior (all other browsers as well) when a website allocates infinite memory in a javascript loop, which can lead to a full system freeze in seconds (since the OOM killer can take minutes to kick in on linux on a graphical desktop, sadly) Often hard reset is the only quick way to solve this, making this a very effective denial of service with almost no way of defending other than advanced sandboxes like VMs.

This could all be solved if the threshold wasn't "oh no malloc() returned NULL" or "99.999999% of system memory and swap used" but some saner lower value, ideally configurable.

Can addons solve this problem? Will kick in fast enogh with a tight website javascript allocation loop to work in all such situations? Isn't it easier to just make this threshold configurable at least through about:config? This doesn't seem like something that should be solved through addon hackery, given the major impact it has on linux desktops.

Can you tell me what the user experience is when a "tab is discarded"?

We close the tabs on them? Or are they suspended in some way? Or do we walk them through a guided path to close it themselves to save on memory? Trying to understand the user experience.

(In reply to Gabriele Svelto [:gsvelto] from comment #187)

Created attachment 9045222 [details]
Bug 675539 - Unload tabs in low-memory scenarios

This adds a mechanism that discards tabs when the browser detects a low-memory
scenario. Tabs are discarded in LRU order prioritizing regular tabs over
pinned ones, pinned ones over tabs playing audio and all of the previous over
pinned tabs playing audio.

discarding is too aggresive

Agree (just tried it). It definitely needs time and memory thresholds options, as for me.

Also need an option "don't unload pinned tabs"

Blocks: 1553260

It should be possible to set Firefox to trigger tab discarding when only free physical memory is running low. User should have a choice to choose between 3 options for browser.tabs.unloadOnLowMemory:
(1) no tab discarding,
(2) tab discarding when the total amount of both free physical memory and swap space is running low (provided in Bug 675539),
(3) tab discarding when free physical memory is running low (default setting) (Bug 1553260).
Tab discarding when free physical memory is running low should be set as default.

Alternatively, values (2) or (3) should be chosen automatically by Firefox based on total RAM present in PC/device: (2) if less than 8 GB is available in PC, and (3) if 8 GB or more is available. This value could be slightly different for different operating systems; user could have option to change this value too.

Discussion:
https://www.reddit.com/r/firefox/comments/brafza/latest_firefox_release_is_faster_than_ever_the/eoc44sk
https://www.reddit.com/r/firefox/comments/bncank/firefox_beta_keeps_reloading_tabs_on_a_pc_with_16/enis26r/?context=5

Thanks for the suggestions but this bug is not the right place to discuss this. Please file another bug to discuss how to evolve this feature and CC me; let's move the discussion over there so we have a clean slate.

(In reply to Gabriele Svelto [:gsvelto] from comment #215)

Thanks for the suggestions but this bug is not the right place to discuss this. Please file another bug to discuss how to evolve this feature and CC me; let's move the discussion over there so we have a clean slate.

I have filed for new bug (Bug 1553260) at the time I made Bug 675539 comment 214.

(In reply to Tradewatcher from comment #185)

This option should not affect tabs with filled forms. This could result in bad UX. (Write a long text - going for a break - all discarded/deleted)

see my comment 91 - form data is already saved (so we can restore it on a crash). note that there is state that will be lost on some pages (think for example a 3D editor without automatic save-to-cloud (or save-to-localstorage)). The Page Lifecycle API is designed to address some of these aspects, and also give the browser more leeway to freeze and/or unload tabs to manage memory and other resources.

Chromium goes through the process of discarding using the Page Lifecycle Spec. https://wicg.github.io/page-lifecycle/spec.html Is this something that Gecko can implement in the future? It may help with the aggressiveness of the discarding because it allows you to remove a certain set of resource usage (gpu textures, decoded images, render trees, etc) while keeping the DOM in a frozen state. When memory pressure then gets too tight it goes into the discard process.

(In reply to dtapuska from comment #219)

Chromium goes through the process of discarding using the Page Lifecycle Spec. https://wicg.github.io/page-lifecycle/spec.html Is this something that Gecko can implement in the future? It may help with the aggressiveness of the discarding because it allows you to remove a certain set of resource usage (gpu textures, decoded images, render trees, etc) while keeping the DOM in a frozen state. When memory pressure then gets too tight it goes into the discard process.

Issues:
https://bugzilla.mozilla.org/show_bug.cgi?id=1480376
https://github.com/mozilla/standards-positions/issues/87

Are also relevant.

Regressions: 1554547
Regressions: 1554124
Regressions: 1558554
Depends on: 1558930
See Also: → 1553260
Depends on: 1475889
Blocks: 1587762
No longer depends on: 1532955
No longer depends on: 1475889

(In reply to Joe Wilson from comment #0)

Since bug 586068 landed - there is a new state for every tab: it can be
stalled (unloaded).

My suggestion is to automatically stall/unload tabs that were inactive for x
hours in order to free some RAM.
The way I see it - there should be a pref in about:config which will define
the time that needs to pass, before the tab (if it wasn't activated during
that time) becomes stalled.
The same thing does BarTab addon.

Did this "pref" ever get done? I think it should be in the options, as opposed to about:config.

(In reply to Worcester12345 from comment #221)

Did this "pref" ever get done? I think it should be in the options, as opposed to about:config.

No, it's also unrelated to this because this one was about unloading tabs to prevent out-of-memory crashes, not to free unused resources. I don't know if we'll add unloading unused tabs even when memory is abundant, but if we do the user should have control over it. In the meantime there's a few extensions that do this and which have tunable parameters (e.g. unload after x minutes).

(In reply to Gabriele Svelto [:gsvelto] from comment #222)

(In reply to Worcester12345 from comment #221)

Did this "pref" ever get done? I think it should be in the options, as opposed to about:config.

No, it's also unrelated to this because this one was about unloading tabs to prevent out-of-memory crashes, not to free unused resources. I don't know if we'll add unloading unused tabs even when memory is abundant, but if we do the user should have control over it. In the meantime there's a few extensions that do this and which have tunable parameters (e.g. unload after x minutes).

Will these extensions work with versions newer than 71.0b9 (64-bit)? What extensions? I might like to try one out.

(In reply to Worcester12345 from comment #223)

Will these extensions work with versions newer than 71.0b9 (64-bit)? What extensions? I might like to try one out.

Give this one a try: https://addons.mozilla.org/en-US/firefox/addon/auto-tab-discard/

There's quiet a few more including ones that integrate with tree-style tabs.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: