Bookpedia and Pocketpedia on multiple devices

[cervantes]

Hi, I have several Macs and several iOS devices.

At the moment Bookpedia runs on one of the Macs and Pocketpedia runs on two of the iOS devices.

Is it possible to "sync" Bookpedia databases between two Macs? If so, how? Can the databases be "two-way synced"?

And if on the same network there are multiple Macs running Bookpedia, what happens when you sync from an iOS device? In particular, what happens if the Bookpedia databases on the Macs are not precisely identical?

In this kind of situation (several Macs running Bookpedia and several iOS devices running Pocketpedia), it would appear that data loss is a real possibility. Are there any protections built into the software?

Thanks.

[Nora]

The whole sync system is designed with a one single Bookpedia in mind. Bookpedia always acts as the master database, and timestamps on the last sync as well as a sync log is used to perform the two way sync. The system is built so that Bookpedia can handle several Pocketpedia's but Pocketpedia can not handle two Bookpedia's as then there are two master databases.

If two Bookpedia's are present on the same network, Pocketpedia will prompt the users to select one and append the computer name to one of them, so that user can tell them apart. If the database are distinct you will get a warning in Bookpedia that you must acknowledge, the warning states that the database will replace all data in Pocketpedia as the original database was not from this Bookpedia.

If the database are copies of each other, then you should have a system in place to sync them without the use of Pocketpedia. Most users user Dropbox for this, keeps the Bookpedia data folder synced and shared via several Bookpedias. This way there is a single master database and it does not matter what running instance of Bookpedia a Pocketpedia syncs to as both are running from the same data folder.

[cervantes]

Thanks for the response.

If the database are copies of each other

Yes, this is what I intend.

then you should have a system in place to sync them without the use of Pocketpedia. Most users user Dropbox for this, keeps the Bookpedia data folder synced and shared via several Bookpedias. This way there is a single master database and it does not matter what running instance of Bookpedia a Pocketpedia syncs to as both are running from the same data folder.

Is there a way to achieve this without Dropbox but using (Mac OS X) aliases instead?

Thanks.

[cervantes]

Continuing the above line of inquiry, I found the following in your (excellent) Help files:

Sharing a database

To share a Bookpedia library between two or more computers, move the Bookpedia data folder to a location where all computers have access to it, like a public folder or a shared hard drive. The move works the same for all users as described in the section above.

If I move the Bookpedia data folder out of its default location (in Application Support) into some other location, will there be undesirable (or indeterminate) side effects?

And if I share the Bookpedia data folder located on computer X with another computer Y on the network, what happens in Bookpedia running on computer Y when the network connection is down? (I am thinking about the possibility of data loss.)

Thanks.

[Conor]

You can indeed share on the network. That was the original intent and why the data folder is movable. Dropbox just happened later as everybody started going on the cloud.

When the connection is down Bookpedia will try to open the network connection to the folder by asking Finder to load the location. If it fails you will get a warning saying it can't load the data folder. There is a default button on this warning message to take you back to the Application Support, some people click it without really reading and then end up at a brand new Bookpedia data folder in Application Support. So there is no data loss, as you can go back to loading the shared version once the network is restored, but can cause confusion. If you are aware that this can happen and know to then reload from the shared location, then there is slightly less nerve racking stress. Either way to eliminate all stress, everybody should always be doing a backup of the data folder on a regular basis, be it with TimeMachine, Superduper! or even a manual copy of the data folder.

You can always check the location that is currently being read in the preference window at the bottom of the window, it will list the path to the data folder.

[cervantes]

Thank you.

I am now sharing the Bookpedia data folder across two (only 2) Macs.

Not sure why when Bookpedia is started on the non-hosting Mac, I sometimes get this error message when the program tries to access the shared folder: "The file cannot be opened." (The program quits immediately after displaying the message.)

Similarly on the hosting Mac I sometimes get this message: "The database in memory is not in a valid state and will be reloaded from the hard disk" — at which point, after I say "Reload," changes made on the non-hosting Mac finally appear. (The error message also appears every two or three minutes even if no changes were made on the non-hosting Mac, or even if the non-hosting Mac is asleep, and the loop can persist until I force-quit Bookpedia.)

Apart from that I have noticed a couple of minor things.

1: Renaming the (shared) data folder requires that Bookpedia be informed again as to its location. I guess this is partly because of how file-sharing works in OS X and partly because Bookpedia looks for the specified folder by name as opposed to ID.

2: Sharing the data folder does not automatically bring user preferences over from one machine to another. Here I am referring to things like "Show Count" vs. "Hide Count" — and also to things like the list and sequence of library servers queried for information about books. I assume (but I have not yet checked) that copying Bookpedia's .plist file from one Mac to another will resolve all such issues.

Anyhow, these are details. I appreciate the overall solution. Thanks again.

[cervantes]

Conor, in the immediately preceding post I mentioned two error messages thrown up by Bookpedia:

1. "The file cannot be opened."

2. "The database in memory is not in a valid state and will be reloaded from the hard disk."

(See the previous post for context.)

Can you offer any guidance regarding these error messages? Thanks.

[Conor]

Hi Cervantes, sorry the previous thread been waiting a while.

The error message appears when the database is not in the format that it should be. Core Data from Apple should have been merging the changes made by both Bookpedia, but it can easily be that locking mechanism allowed both to write at the same time confusing the database. This al depends on OS X and the way the network is mounted as different file sharing mechanism are used by the OS, way under Bookpedia.

I can take a look at the file and try to fix it, if you send me the "Database.bookpd" file from the data folder. Otherwise if you have a recent backup you can replace this file with that backup or with one of the automatic backups "Backup1Month.bookpd" in the same folder that we never actually use, but create so in case since the file is so small. Without Bookpedia running, you can rename the "Backup1Month.bookpd" to "Database.bookpd". Once backup and running, do keep frequent backups of the database file, as well as seeing when the error occurs again, to know what specific simultaneous edits Bookpedia and OS X is having trouble handling.

Indeed the preferences are not shared, you need to copy the preference file from one machine to the next to set the same preferences. Yo will find the file in the user Preference folder at ~/Library/Preferences/com.bruji.bookpedia.plist

[cervantes]

No problem re the delayed response.

No problem also re data loss: I was only testing this arrangement, no actual data were harmed!

Not least for my own benefit I would surely like to help figure out what's wrong — but this hope may not be realistic. Regardless of where the fault lies, if the sharing mechanism is not something I can rely on, I may not have more time to invest in it. But if I do further testing, and if something useful emerges, I will certainly let you know.

One thing I forgot to mention: Of the two machines in question, the host Mac is running Lion and the other Mac is running El Capitan (latest versions in both cases). Maybe this information helps?

[FineWine]

Conor is this what you have been waiting for re better integration between iOS and Mac OS pedia's: http://9to5mac.com/2016/02/05/apple-add ... d-service/

[Conor]

Thanks for the all the info and testing, if you have a Dropbox account, you might want to test that out as well. It has the added benefit of being error correcting as Dropbox tracks all changes and keeps a version of the previous file, that you can revert to should merges tang each other up. In truth Bookpedia still leaning to much on the speed side and not watching and reloading the file often (lots of caching going on) the next version will be more attentive to changes and be more robust to simultaneous use, as computers have gotten fast enough that the caching is not a big a win as it was a few years ago. Specially with more and more users on SSD drives.

Thank you for the link, the new API does look interesting, as well as the reworked iCloud Drive and better error handling in El Capitan of said iCloud documents. I have yet to dive into all the new changes in depth, but have it all bookmarked.

[cervantes]

Two questions:

Is there a way to specify a master database on the network without Dropbox and without file-sharing — but using (Mac OS X) aliases instead?

And when you say:

Bookpedia always acts as the master database, and timestamps on the last sync as well as a sync log is used to perform the two way sync.

I've been meaning to ask: The time-stamps appear to pertain to entire records; whereas have you considered implementing time-stamps on the level of each field in a record, or would that be a bad idea for some reason?

Thanks.

[Conor]

but using (Mac OS X) aliases instead

Not sure how you would use aliases without also using file sharing, there has to be access to the final data location for the program to be able to read the data.

The time stamp per field was considered, but just made the database to big to keep track of all that information with little gain of just more fine grained sync. So a timestamp is kept per record and values merged, when values differ the one with the nearest time stamp wins.

[moses]

If the database are copies of each other, then you should have a system in place to sync them without the use of Pocketpedia. Most users user Dropbox for this, keeps the Bookpedia data folder synced and shared via several Bookpedias. This way there is a single master database and it does not matter what running instance of Bookpedia a Pocketpedia syncs to as both are running from the same data folder.

If I am syncing my database between two (or more) Macs via Dropbox, what happens if multiple copies are open at the same time and edits are made on each? Can it handle Dropbox syncing updates between the Macs in the background while in use?

Or, is it best to only actively update the database from one Mac at a time?

Am looking for the best way to share a database between multiple Macs and concerned about conflicts.

[Conor]

Most of the time it will be able to merge the changes. But The Pedia does a lot of caching so you might not see the changes made on the other machine until you restart the program. In general if you can, we recommend that you quit the program, so that you are mostly only using one at a time. Once we drop support for the older MacOS, I will be able to add a feature to detect file being updated and refresh the changes right away.