Holly Becker's Blog

How to buy Japanese books (from Kobo)

2024-05-16T00:00:00Z

I managed to exchange money for books in Japanese!

I've found it very hard to buy Japanese language media from outside Japan, and ebooks in particular are hard to find in other ways. I have an eReader I like reading on, which means I don't want to use the bookseller's mobile app to read a book, which cuts out most of the options.

My eReader is made by Kobo, which is owned by Rakuten, which is a Japanese company. This should be easy! I tried using my existing Kobo account to buy from the Japanese Kobo store, but even with a VPN "Japan" didn't appear in the location dropdown. Some searches revealed that you can only say you're in Japan if you log in with a Rakuten ID. The Rakuten ID reportedly can't see Overdrive (which I use to check out library books in Canada), and switching accounts involves resetting the entire device, so that was a dealbreaker for me.

Time passed.

My Kobo became flakey (but not unusable) so I got a new one. Now I had a spare mostly-working eReader to mess around with, remembered the above problem, and tried it again.

Creating a Rakuten ID was straightforward and free. Despite the link being in Japanese (楽天会員登録(無料)) the sign up form itself was in English, which I think was the only part of this process I did in English. I was able to skip entering an address initially, but when I actually bought a book it required one, so I used a hotel that I'd stayed at on the assumption that they won't check and it likely won't matter. I browsed Kobo's Japanese site, found a book I wanted to purchase, and bought it with my Canadian credit card. I set up my new Rakuten ID account on the eReader, voila! my newly purchased book was there!

After how much hassle it was the other times I tried and failed, it was almost a letdown that it was this easy, now that I had a dedicated Japanese reading device.

The case of the missing 32-bit Ruby package

2023-08-03T00:00:00Z

I have access to a 32-bit Arch server that hosts this website and I use Ruby on it to run jekyll for static site generation. I don't touch it very often, but recently I started getting this exception:

$ irb
irb(main):001:0> require "date_core"
<internal:/usr/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- date_core (LoadError)
        from <internal:/usr/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from (irb):1:in `<main>'
        from /usr/lib/ruby/gems/3.0.0/gems/irb-1.4.2/exe/irb:11:in `<top (required)>'
        from /usr/bin/irb:25:in `load'
        from /usr/bin/irb:25:in `<main>'

I reproduced it with just the require, but I first encountered it running bundle exec jekyll build. The error is happening because Ruby is trying to load date_core (which is a compiled-from-C .so file) and can't find it on any of the load paths. It's the same error you get if you try to import nonsense like require "aoeuaoeu".

tl;dr This can be worked around by setting Ruby's $LOAD_PATH to explicitly include the directory

export RUBYLIB=/usr/lib/ruby/3.0.0/x86-linux/

but it's weird that it's happening at all.

I eventually tracked the problem down to date_core.so being in /usr/lib/ruby/3.0.0/x86-linux/ which was not on Ruby's path so it couldn't find it to load it.

$ ls /usr/lib/ruby/3.0.0/x86-linux/date_core.so
/usr/lib/ruby/3.0.0/x86-linux/date_core.so   # This is in the directory x86-linux
$ ruby -e 'puts $LOAD_PATH'
/usr/lib/ruby/site_ruby/3.0.0
/usr/lib/ruby/site_ruby/3.0.0/i686-linux
/usr/lib/ruby/site_ruby
/usr/lib/ruby/vendor_ruby/3.0.0
/usr/lib/ruby/vendor_ruby/3.0.0/i686-linux
/usr/lib/ruby/vendor_ruby
/usr/lib/ruby/3.0.0
/usr/lib/ruby/3.0.0/i686-linux  # The directory it's looking in is i686-linux

The path that Ruby does look in also exists and has other installed Ruby packages in it. It's very strange that 32-bit .so files for Ruby packages are in both i686 and x86-linux and that Ruby only seems to know about one of them.

$ ls /usr/lib/ruby/3.0.0/x86-linux/
bigdecimal.so  cgi  date_core.so
$ ls /usr/lib/ruby/3.0.0/i686-linux/
continuation.so  dbm.so  digest.so  etc.so    fiber.so   gdbm.so  json        nkf.so       openssl.so   psych.so  racc      rbconfig.rb  ripper.so  stringio.so  syslog.so
coverage.so      digest  enc        fcntl.so  fiddle.so  io       monitor.so  objspace.so  pathname.so  pty.so    rbconfig  readline.so  socket.so  strscan.so   zlib.so

For each directory, I found a package that owned files there so I could look at their build scripts and compare them: ruby-date & ruby-digest.

$ yay -Qo /usr/lib/ruby/3.0.0/x86-linux/date_core.so /usr/lib/ruby/3.0.0/i686-linux/digest.so
/usr/lib/ruby/3.0.0/x86-linux/date_core.so is owned by ruby-date 3.2.2-4.0
/usr/lib/ruby/3.0.0/i686-linux/digest.so is owned by ruby-digest 3.1.0-5.8

Arch Linux has officially deprecated 32-bit support, so this is where the research became harder. I think ArchLinux32 use the same packages as ArchLinux unless something needs to be different. I didn't find anything suspicious in the 32-bit specific packaging scripts so I looked at the upstream PKGBUILDs.

I think both packages use local _platform="$(gem env platform | cut -d':' -f2)" to generate the platform-specific version, but it's producing different results. Since previously I didn't get the exception, I looked in the pacman logs to find the version difference - ruby-date (3.2.2-3.6 -> 3.2.2-4.0) which means it was only a change in how it was packaged - but both before and after use gem env platform so it's probably not that.

I couldn't find a definite answer, but my suspicion is that the different packages (and package versions) are built on different computers, and on some of them gem env platform produces i686 and on others x86-linux, but I'm not sure why.

I didn't get any further than this - the server was reinstalled with a 64-bit Arch which didn't have the same problems. If you run into the same problem or figure out what was going on, please let me know!

Photo rights & Terms of Service

2020-06-20T00:00:00Z

I take a lot of bird photos, and while I enjoy uploading them to eBird for inclusion in their scientific collection, I also like showing them off to friends. I've been uploading them to Flickr with a CC BY-SA-NC license, but since nobody browses Flickr anymore I was hoping to share them somewhere they'd get more attention. However, I also know social media sites are infamous for their aggressive rights grabs on user content, so I read several Terms of Service trying to figure out which was least-bad for uploading photos to, including Dreamwidth, Flickr, Twitter & Facebook.

The relevant sections of the ToS were surprisingly short.

Dreamwidth (https://www.dreamwidth.org/legal/tos)

By submitting Content to us for inclusion on the Website, you grant us a world-wide, royalty-free, and non-exclusive license to reproduce, modify, adapt and publish the Content, solely for the purpose of displaying, distributing and promoting the contents of your account, including through downloadable clients and external feeds.

Flickr, owned by SmugMug (https://www.flickr.com/help/terms)

You retain all intellectual property rights in and to any User Content you post .... [You] grant SmugMug a perpetual, nonexclusive and royalty-free right to use the User Content ... as is reasonably necessary in order to do the following: (1) provide the Services, including to display the User Content on the Services (2) ...

Twitter (https://twitter.com/en/tos)

You retain your rights to any Content you submit .... [Y]ou grant us a worldwide, non-exclusive, royalty-free license (with the right to sublicense) to use, copy, reproduce, process, adapt, modify, publish, transmit, display and distribute such Content in any and all media or distribution methods now known or later developed (for clarity, these rights include, for example, curating, transforming, and translating).

Facebook (https://www.facebook.com/terms.php)

You own the intellectual property rights ... in any such content that you create and share on Facebook and the other Facebook Company Products you use. .... [W]hen you share, post, or upload content that is covered by intellectual property rights on or in connection with our Products, you grant us a non-exclusive, transferable, sub-licensable, royalty-free, and worldwide license to host, use, distribute, modify, run, copy, publicly perform or display, translate, and create derivative works of your content

Some of these are reasonable or expected:

"worldwide" = the internet is everywhere
"non-exclusive" = you can still grant licenses to other people or companies
"royalty-free" = they're not paying you for the photos
"to use, copy, host, distribute" etc = they need to store & display the photos to run the service

Others are a lot worse:

"transferable" = company can give the license that you gave to the company to someone else, and you have no say in it
"sub-licensable" = company can license your content to another person or company, including charging money for it, and you have no say or right to the money

Of these four, Flickr & Dreamwidth are the best, with the smallest rights and strong limitations on them. Twitter has the 'sublicense' clause, and Facebook has both 'transferable' and 'sub-licensable', both of which are pretty bad. This doesn't surprise me - Flickr is about photo sharing from people that might want to sell their photos and Facebook is rife with other problems - but still interesting to see that reflected in the ToS.

I ended up continuing to upload my photos to Flickr under a Creative Commons licence, and linking them from Facebook, relying on the preview functionality.

Can I Use That Code? Software licences as a user

2017-11-26T00:00:00Z

I did my first conference talk for PyCon Canada last weekend on software licences¹ as a end user. I had given this talk before to a few meetups and it had gone well, which encouraged me to submit it to PyCon Canada 2017 and PyCascades 2018. To my delight (and nervousness!) both conferences accepted it!

I really enjoyed the experience. While I was very nervous, and was finishing my slides and practicing fervently the night before I presented, the talk itself went fine. As a bonus, I've had several people tell me they enjoyed it, and I have an excuse to talk about software licences to everyone!

While most of the content can be found elsewhere on the internet, I know many people (myself included) prefer reading over watching a video so I've provided a summary/transcript. Slides are also available.

tl;dr

Introduction

I'm going to try to answer the question "Can I use that code?". I'll be talking about software licences as a user, focusing on looking at other people's code and checking if you can use it in your project

I'm Holly Becker. I'm a software developer at Sauce Labs, which does automated mobile and web testing as a service. I'm also a PyLadies Vancouver co-organizer and a fan of open source. Aside from programming, I enjoy books, board games and birdwatching. You can find me online as Hwesta.

Another important thing about me is I Am Not A Lawyer. I did some research for this presentation, and while it was fascinating and informative it is definitely not legal advice. You should consult a lawyer if you have any actual legal concerns. My research isn't enough because...

...software licences are complicated. There are lots of interactions and complexities; some requirements conflict; there's unclear definitions of terms, or terms from legal precedent from another country. Much of this has not been tested in court, so the final decision may be different than we think it is (or should be). This is just a summary to get you thinking about software licences.

What is a Licence?

First of all, what is a licence?

Specifically, what is a software licence? I'm only going to discuss software licences. If you have other creative non-code work, I highly recommend the Creative Commons set of licences. Licences discuss what you can do, and what you must do to be able to use the code. It allows uses that default copyright does not, and addresses how you can run, modify and redistribute the code (for example, using it in your project).

Categories of Licences

There are lots of different software licences. Fortunately, they can be grouped into a few categories with similar rights and obligations, which I'll cover from most restrictive to least restrictive.

The first category is code that is not licensed. This is not actually a licence! By default everything is under copyright and not usable by anyone except the copyright holder, usually the author. You can ask for permission to use it, but unless it has a licence you can't use it. This includes the great looking code you found on someone's blog, or that useful website you found.

The next category is closed source or propriety licences. Generally, you can't use this code, or you can only use it in a restricted sense like running but not modifying, or you have the compiled binary but not the source code. A common example of this is a EULA, where you agree to the terms and conditions in order to run the program. Another interesting example is the Unity game engine; you can use it for free to make video games until your revenue passes a certain amount, then you have to pay to use it.

The first open source category is copyleft licences. You can use this code! However you have some significant obligations. You have to keep a copy of the licence with the code, and derivatives have to be licensed under a copyleft licence. A derivative can be modifications to the original code, or a new program built with the code, but both of them have to be released under a copyleft licence. Copyleft is concerned not just with making software free but with keeping software free and the licences in this category enforce that. The best known example of a copyleft licence is the GPL, and the related AGPL and LGPL.

The second open source category is permissive open source licences. You can use this code! There are very few restrictions and generally just require distributing the licence with the code so others know the terms the code is available under. Most licences also disclaim liability and may have additional restrictions such as granting patent licences or say that using this code isn't an endorsement. Unlike copyleft, permissive licences are not concerned with derivatives' freedom and can be used in closed source applications with few issues. This is a very popular category of license - more than half the projects on PyPI are licensed under one of the permissive open source licences. There are several well know licences in this category, including the MIT, BSD and Apache licences.

Code can also be put in the public domain. You can use this code! There are no restrictions, no credit required, no licence to keep with the code. Notably, there is no copyright. This is the biggest difference to the previous licences. In fact, it's not actually a licence, because no one has copyright. Licences work by using copyright to control usage, even if the rights granted are broad and the conditions minimal. If there's no copyright, then there's no one to set the terms, so no licence is possible. There's also no disclaimer of liability, so you probably want a permissive open source licence instead of public domain for your code. However it has been done: the SQLite database is public domain. The "licences" in this category (CC0 and the Unlicense) offer public domain-like rights in jurisdictions with no concept of public domain.

The last category is joke licences. This is not really a useful category. The intent is to be both permissive and funny, but it ends up being really legally problematic. For example, "You can use my code if you buy me a beer!". I buy you a beer? My company buys you a beer? Do all my coworkers have to buy you a beer? Do we buy you a beer before or after we start using your code? Do we have to hunt you down for the beer buying? This is problematic for lawyers and they have to buy themselves a beer to deal with it. You should avoid libraries with these licences because it's a legal headache, and you should avoid licensing your code under a joke licence.

Common Licences

Generally if you're using someone else's code it's available under one of the open source licences. We'll look at a couple permissive and copyleft licences to set the stage for discussing how they interact.

Starting with the shortest, the MIT Licence is a simple permissive open source licence. It only requires keeping a copy of the licence (which includes the copyright holder's information) with the code. Like basically every licence, it disclaims liability. It's actually quite short, only about 170 words, and there's an excellent breakdown by (/dev/lawyer)devlaywerMIT explaining all the legalese. Some Python packages licensed under MIT are SQLAlchemy, Twisted and six.

The BSD licence is very similar to the MIT licence: a simple, permissive open source licence. It disclaims liability and says you must keep a copy of the licence with the source or binary. There's two major versions of the BSD licence: the 2-clause and 3-clause. The 3-clause adds another clause saying that copyright holders don't endorse any derivatives. This makes more sense in the context that BSD is short for Berkeley Software Distribution, and the University of California in Berkeley probably didn't want people saying Berkeley endorsed them just because they were using software written at Berkeley. This as a popular license: it's used by scipy, numpy and pandas from data science/scientific computing, and Django and flask from web programming.

The Apache licence is the last of the permissive open source licences I'll discuss. It's longer and more explicit about things the previous licences are implicit about. Like the others, it includes a disclaimer of liability and you must keep a copy of the licence with the code. It also explicitly addresses patents, and grants the patent rights necessary to run and distribute the code. Amusingly, these rights are revoked if there's a lawsuit over them. Additionally, contributions to Apache licensed projects are by default licensed under Apache themselves, which is an interesting clause I didn't see elsewhere. The Python packages requests and selenium are licensed under Apache, as well as the Android operating system.

And last, but definitely not least, is the GPL or GNU Public License. This is the best known of the copyleft licences. It includes the things you're probably coming to expect from a software licence: disclaimer of liability, keep a copy of the licence with the code. Like the Apache licence, version 3 includes a grant of patent licence.

On top of that, the GPL has all the things that make it a copyleft licence. If you distribute code (or modify and distribute code) licensed under the GPL, you must make the source code available. Note that if you only use it for personal use or internal to your company you don't have to do anything special. It's the act of distributing it (either giving away or selling) that requires you to make the source code available.

Additionally, if you include GPL licensed code in your codebase and distribute it you must make the entire codebase available under the GPL. Remember, copyleft licences are concerned with keeping software free; they don't want you to make something with their code and not contribute it back to the community. The GPL is sometimes called 'viral' because including GPL'd code affects the licensing of the entire program.

However, the definition of 'one program' gets complicated. What's one program vs two programs that communicate? Typically static and dynamic linking are considered one program, while HTTP calls and piping output are not. In between things get fuzzy. This is where the lawyers get involved if you want to use a GPL licensed dependency but don't want to release your codebase.

The best known example of a GPL licensed project is the Linux kernel, and the Python project Ansible is also licensed GPL.

Can I Use That Code?

This is the interesting part! Now that we know a little about the licences, we can discuss if my code is licensed X can I include code licensed Y. In the presentation, I asked people to guess the answer before I revealed it.

Closed Source + MIT: Yes! MIT has few restrictions, only requiring a copy of the licence be kept with the code so others know how it's licensed, and can easily be used in closed source projects.

Closed source + GPL: Probably not. There are workarounds to ensure your project and the GPl'd project are considered separate projects, but you're probably going to want to talk to a lawyer to make sure you get it right.

GPL + MIT: Yes! For the same reasons MIT licensed code can be used in a closed source project: the MIT license is permissive and has few restrictions. Note that while the dependency is licensed MIT, the combined work of your project and the dependency is distributed under the GPL, because that's what the GPL requires. (I did say licences were complicated.)

GPL + GPL: Yes! Two pieces of code with the same licence is rarely a problem because they have the same terms which are unlikely to conflict.

MIT + MIT: Double yes! They're the same licence, and the MIT licence is very permissive.

MIT + GPL: Probably not, for the same reasons it's difficult to include a GPL'd dependency in a closed source project. The GPL requires that derivatives be licensed under the GPL as well. You could license your project GPL, but if you want to retain the MIT license it requires the same sorts of careful separation from the less-restrictive MIT license to prevent the GPL's viral nature from overriding it. You're probably going to want to consult a lawyer.

Common questions

There's a couple things that always come up when discussing licences.

Can you use code on Github? By default, code on Github falls under unlicensed and you can't use it. The copyright holder has given Github the rights to display the code and a few other things like forking, but that doesn't mean you can use it. However, many repos have licences, and you can use that code under the terms of the licence. In short: check!

What about Stack Overflow? Stack Overflow is complicated. User contributions are licensed CC-BY-SA, which is a copyleft licence for non-code works. However, it's not clear that Creative Commons licences can apply to code. Usually the text is licensed under CC and the code is separately licensed under a software license. Stack Overflow discussed licensing code under the MIT license (or a modified MIT license), but it failed to happen because of community push back over the suitability of the license for tiny snippets, whole programs and code in questions. This is also complicated by the fact that not all snippets of code are big enough to be covered by copyright. To be covered by copyright, and therefore be licensable, it has to be creative. Since many small snippets of code are common ways of doing things and not creative, copyright may not apply. Overall, it's a very awkward situation. I won't make a recommendation, but I will admit I use SO for inspiration and reference.

Conclusion

There's a few things I hope you remember from this.

Software licences are fantastic! They allow you to use other people's code and allow others to use your code, enabling the whole open source community.
Software licences are also complicated
Permissive open source licences are simple, straightforward and easy to use
Copyleft open source licences take a strong ethical stance. It doesn't want code being used without contributing back to the community.

But most of all, remember to check the licence! Just because code is on the internet doesn't mean you can use it.

When making this talk I got to learn the difference between licence (with a c) and license (with an s). In American English, they only use license. In non-American English, licence is the noun (think advice) and license is the verb (think advise). Grammarist has a good explanation.↩

邪魔 (じゃま) and cats

2017-10-29T00:00:00Z

When I encounter a word I don't know in Japanese, one of my techniques is to use google.jp to image search for it. The goal is to keep from translating the word into English and to get a sense of what it means in Japanese instead.

The other day I came across 邪魔jyama and googled it.

Thought process: Hmm, what does this mean? Something about cats sitting on keyboards? That doesn't seem right from the source context. What's that in the bottom right? I recognize that board game, it's Saboteur!

So 邪魔じゃま might mean sabotage or obstruct or hinder, but it's apparently quintessentially cat-on-keyboard!

TIL: SASS & locale-gen

2017-10-22T00:00:00Z

I decided to switch this site from a handful of HTML & CSS pages to a static site generator, Jekyll. When deploying it to my webserver, I ran into some locale problems with SASS generation and thought someone else can learn from my pain.

Just the solution please

I've never used Jekyll before and I haven't really used Ruby so I had a few problems installing gems into the right places that I sorted out with Misty's help. Jekyll, Bundler & SASS all recommend gem install. Jekyll uses Bundler for project dependencies. However, I later ran into what I think were conflicts between the gems installed in my user directory and the gems installed by Bundler for my project. My solution was to install Bundler & SASS at the system level (pacman -S ruby ruby-bundle ruby-sass), use Bundler for project dependencies, and use bundle exec jekyll for any Jekyll commands I needed. The rm -rf ~/.gem to delete user directory gems was very cathartic.

The conversion to Jekyll was pretty easy and I was ready to deploy it to my webserver. When trying to generate the site on the webserver I ran into this problem:

$ bundle exec jekyll build
Configuration file: /home/holly/hollybecker.net/_config.yml
            Source: /home/holly/hollybecker.net
       Destination: /home/holly/hollybecker.net/_site
 Incremental build: disabled. Enable with --incremental
      Generating... 
  Conversion error: Jekyll::Converters::Scss encountered an error while converting 'css/main.scss':
                    Invalid US-ASCII character "\xE2" on line 2
jekyll 3.6.0 | Error:  Invalid US-ASCII character "\xE2" on line 2

Huh?

\xE2 is â but I don't have any of those in my project. It's definitely not on line 2 of css/main.scss since that's the second line of the empty YAML frontmatter needed so Jekyll parses the file. Also, this command works on my desktop and another Arch Linux server. The versions of SASS, Jekyll and Bundler are the same. The webserver is 32-bit, but a 32-bit VM Arch Linux VM didn't have this issue either.

Google led me to a Jekyll issue with the same problem I was having which was quite useful. It suggested specifying the encoding in several places, none of which worked. It also helped track down that the problem wasn't in css/main.scss but in the imports and provided a neat tip on how to grep for non-ASCII with grep --color='auto' -P -n "[\x80-\xFF]" file.xml

A-ha! Grepping for non-ASCII on the webserver found a lot of comments with en-dashes instead of hyphens. An en-dash is \xe2\x80\x93, hence the misleading \XE2 in the error earlier. However, the same grep command on my desktop didn't find anything.

Now I had a workaround - I could replace the en-dashes with hyphens - but I was curious what's going on.

I narrowed it down to a more minimal test case using the pre-Jekyll master branch. The problem was just with SASS, not Jekyll or Bundler. I hadn't run into it previously because I ran SASS on my desktop and committed the results, instead of generating the site on the webserver.

$ sass sass/main.scss css/main.css 
Error: Invalid US-ASCII character "\xE2"
        on line 2 of sass/base/_variables.scss
        from line 14 of sass/main.scss
  Use --trace for backtrace.

This is clearly an encoding problem, so I looked at the locale related environment variables. My desktop looks fine:

holly@desktop$ env | grep LANG
LANG=en_CA.UTF-8
LANGUAGE=en_US:en_GB

But the webserver is very suspicious:

holly@webserver$ env | grep LANG
LANG=C

The C locale is aimed at computers over people and doesn't understand UTF-8. However, setting the locale to something with UTF-8 support (export LANG=en_CA.UTF-8) didn't fix the problem.

I didn't realize this until the next morning, but I'd forgotten to check which locales were actually generated. Neither en_US.UTF-8 nor en_CA.UTF-8 actually existed on the system and it was probably falling back to the only locale it knew about: C.

To fix this I generated the en_US.UTF-8 locale (uncommented en_US.UTF-8 UTF-8 in /etc/locale.gen and ran locale-gen) and confirmed it worked by listing all generated locales (locale -a). I set the system locale in /etc/locale.conf to LANG=en_US.UTF-8, though once the locale was generated, I could have set the LANG in the terminal I was using as well.

After all this, it worked! Jekyll & SASS ran and correctly generated the CSS, even with unicode characters involved.

As for why this happened, this webserver had been installed in 2009 before Arch changed the default locale to one with UTF-8 support. All the other servers I checked on had been installed more recently or had been updated to a UTF-8 locale and didn't have this problem.

Summary

When generating SASS, I had the error:

$ sass sass/main.scss css/main.css
Error: Invalid US-ASCII character "\xE2"
        on line 2 of sass/base/_variables.scss
        from line 14 of sass/main.scss
  Use --trace for backtrace.

The fix was to generate UTF-8 system locale and set it.

Check if locale exists: locale -a
If not, generate locale: Uncomment locale name in /etc/locale.gen and run sudo locale-gen
Set locale for the system: Edit /etc/locale.conf and log in again for it to take effect
Or set locale for the terminal
- bash: export LANG=en_US.UTF-8
- fish: set -xg LANG en_US.UTF-8