Firefox Sync creates duplicate Bookmarks
I just setup Firefox Sync on two (2) of my systems and configured it to just sync Bookmarks. The existing bookmarks on these two (2) systems were already almost the same, but one (1) system had not been used recently so it didn't have the latest bookmarks I had saved on the other system.
Unfortunately, after sync did its thing, I now have many, many duplicate bookmarks. Doesn't sync check and eliminate duplicates between the two (2) systems?
Doing a sync reset won't fix the issue, because each system and the data on the sync server contains the duplicates. Is there a way to tell the system to de-duplicate what is on the sync server and then update each of my systems with this newly cleaned data from the server.
Any help is appreciated.
Thanks, Gary
All Replies (14)
rmcguigan,
Thanks for the reply. This addon would be somewhat helpful. However, because of the number of Bookmarks I have, it would still be a very manual and time consuming process so it is not really a solution. The addon says I have 7966 duplicates.
If it is the same link and in the same location (folder), then it is a duplicate and needs to be removed. Somethings to automatically do this would be great.
Of course Firefox Sync shouldn't have created the duplicates to start with. Isn't this a bug? Shouldn't someone be fixing it and maybe creating a function to de-dup what is on the server.
Thanks, Gary
If you still have a JSON bookmarks backup that isn't effected (Firefox keeps 10 in the bookmarkbackups folder) and you haven't added new bookmarks recently that would get lost then you can consider to restore the bookmarks from a JSON backup.
Hi gtdriski,
Currently on each update there has been this issue reported and a little more information would be really valuable to help find the root cause.
If possible could you turn on services.sync.log.appender.debugLog.enabled to true in the about:config page.
Please navigate to about:sync-log and view the logs for any errors around the time that the bookmarks were noticed to be duplicated.
rmcguigan,
I would love to help. Unfortunately, I don't see services.sync.log.appender.debugLog.enabled on my about:config page. I did a search and it found nothing. The only related Preferences are shown in the attached image. Is there a way I can add it?
Thanks, Gary
Gewysig op
Ok, the reference I found is from a different version of Firefox, my apologies on that.
Hi sorry for the delay,
I did find some other questions with similar issues that were due to extensions:
Some reset, some reinstalled, but this will loose all of the data. If you have a system restore backup, the places.sqlite file in the Profile can be manually restored to before the duplicates as well.
Was Xmarks on when this happened?
rmcguigan,
First I want to say I really appreciate you working on this with me.
I don't really want to do a reset, as that will cause me to loose data as you mentioned. I'll check on the system restore, but I may not have a recent one.
Xmarks was installed, but has been disabled for a long, long time.
I really think the solution is at the Sync Server level. Many people have mentioned deleting the duplicates, but they reappear. This is because the master data set on the Sync Server contains the duplicates. When I first setup Sync on my first PC, the data set on the Sync Server was 3.2MB. After I added the second PC, that data set became 6.1MB, so it obviously has duplicates.
If we had a tool/function to de-dup the master DB on the Synce Server and then overwrite each PC with this newly de-dupped DB we could fix this. Otherwise, I don't see a solution other that turning Sync off and manually cleaning my Bookmarks. What a mess.
Thanks, Gary
Hi gtdriski,
I will let others answer aspects relating to apparent sync problems and bugs.
There are two things you should try the first is URGENT
- Back up your profile immediately
Back up and restore information in Firefox profiles - Try the places maintenance addon
https://addons.mozilla.org/firefox/addon/places-maintenance/- Before; then after running; the repair attempt, run the option to create a report log (They are short text files and may be pasted into your reply.)
- Note It may just fix the problem, it is unlikely to do any harm, and even if it did step 1. above gives you a method to reverse this
- Run the report log, use the repair options offered, then repeat the report logging.
You need to be very careful with bookmark restores as they over write and so destroy existing bookmarks, you should therefore always manually backup your bookmarks (Or make and retain a copy of the places.sqlite database) before any restore attempt.
The reason I say backing up the profile is urgent is because it contains snapshot files of your bookmarks that will be recycled and if this problem has only happened a few days ago you need to preserve them now, because they may be very useful, but will not be around for long.
Hi Gary,
Not a problem, we love it.
Agreed, may I encourage you to vote on or file a bug if one does not already exist, whoops ok so I found some:
- Sync tracking bug for duplicate/removed/reordered bookmarkshttps://bugzilla.mozilla.org/show_bug.cgi?id=621584
- https://bugzilla.mozilla.org/show_bug.cgi?id=950167
Ah also quick question what is the option you have selected in sync to handle merging once the previous sessionrestore.bak is found, or if it is found https://support.mozilla.org/en-US/kb/how-do-i-manage-my-firefox-sync-acco...
- set the .js file with no duplicates in the profile folder
- Make sure all the bookmarks are not duplicate
- and resetting sync with merging devices with that one that is not duplicated should get you up an running.
Mozillazine extracting current urls from sessionrestore discussion: https://support.mozilla.org/en-US/kb/how-do-i-manage-my-firefox-sync-acco...
Maybe this add on is better? https://addons.mozilla.org/en-US/fire.../bookmark-deduplicator/
Rachel,
You repeated the link
- https://support.mozilla.org/en-US/kb/how-do-i-manage-my-firefox-sync-account#w_managing-your-sync-data
- ( or general link How do I choose what information to sync on Firefox?_managing-your-sync-data )
Did you intend something different for
- Mozillazine extracting current urls from sessionrestore discussion:
- Possibly http://forums.mozillazine.org/viewtopic.php?f=38&t=622036&start=75
- That's a locked thread and a long read, 80 posts
Note
There is another possibly related thread
bookmarks multiply every day /questions/980733
Indeed John99, thank you!
Rachel and John,
Here is where I stand:
- I backed up the profiles on both my machines
- In the Sync options, on both machines, I unchecked "Bookmarks". This was all I had been trying to sync.
- I viewed the sync quota and it was 0. Therefore, the sync server should be empty.
- I was able to restore the bookmarks on both machines from backups, so we are back to where we were before the Sync attempt.
- On my primary machine I checked "Bookmarks" again in the Sync Options.
- Still on the primary machine, I selected "Reset Sync" and chose the "Merge this device's data with my Sync data" option.
- I then viewed the quota again and it is 3.2mb. This should be just the one machine's bookmarks with no duplicates.
- I have done nothing else on the second machine. "Bookmarks" is still unchecked in Sync.
In an attempt to get Sync to work correctly with the two (2) machines, how do you recommend I proceed from here?
Thanks, Gary
Hi Gary
Thank you for your message. If the account has selected to sync bookmarks on a specific device, only that device will sync the bookmarks from the synced data on the sync server, or the selected server that the account is syncing to.
I just did an experiment, I unselected the second device I have paired to my primary device and unselected bookmarks in the Sync Preferences. I then added a bookmark and forced sync now. On the primary device the bookmark, xkcd: Buzzfeed Christmas, did not show up. But I then went back and selected Bookmarks and forced synced and the bookmark did show up. And there were no duplicates.
However if the second device is a mobile! This will sync into another Bookmark folder called Mobile Bookmarks. I added the oatmeal, an online comic to the mobile bookmarks and after I synced there were in fact two bookmarks of the same page in the bookmark toolbar, but!!!! in the search for the bookmark it only showed up once.
I am not sure where the file size you are looking at is changing, this may be by design, however this does not mean the duplicates are normal. So from where you are selecting bookmarks on the second device will not create duplicates but if I add adding it by merging the data after resetting it does not create duplicates.