Every once in a while, I need to take lots of pictures, which then need to be mass-uploaded to Appropedia. Doing this all one at a time is tedious. Therefore I wrote a small python script to do it for me.
The script will log in, go to the file upload page, and send an image with given description, then it will repeat the image sending until all files are done. Unless special treatment is given to you by the administrators, you will be regularly blocked by a captcha. The script does not try to handle this.
System requirements[edit | edit source]
The script is written for use on a Debian GNU/Linux machine. It should work fine on anything that runs Python. However, it needs to be run from the commandline (with environment variables set for a comfortable experience). Windows users may find it very hard to do this.
Using the script[edit | edit source]
The script must be run from the commandline. It takes the files to upload as arguments. Optionally, you can pass your login credentials as an argument as well, but normally you will want to pass them through an environment variable (see below). It is also possible to pass a description on the commandline; it can also be set with an environment variable. All images will have the same description, and will be published using the GFDL.
Example[edit | edit source]
To upload some photos of a setup, you can use the following commands:
export APPROPEDIA_NAME=wijnen export APPROPEDIA_PASSWORD=password export APPROPEDIA_UPLOAD_DESCRIPTION='[[Category:MOST methods]] I took this image.' ./multi-upload photos/MOST_HSPrusa_photo*.jpg
Environment variables[edit | edit source]
The commands to export the variables only have to be given once for each shell. It is easiest to set them in your shell resource file, which is automatically loaded at startup. In most cases, this will be a file named '.bashrc', which is located in your home directory. Add the three lines to it, and you never need to type them again.
The script[edit | edit source]
Since it's not possible to upload plain text files to appropedia, I'll copy the script here (which I release CC-BY-SA by posting here, in addition to AGPL-3+, as the header says):
#!/usr/bin/python
# vim: set foldmethod=marker :
# {{{ Copyright 2013 Bas Wijnen <wijnen@debian.org>
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# }}}
# {{{ Imports.
import sys
import os
import urllib
import urllib2
import re
import mimetools
import mimetypes
import argparse
# }}}
# {{{ Commandline arguments.
a = argparse.ArgumentParser (description = "Upload multiple images to appropedia. Make sure their filenames are as you want them in appropedia. If you don't supply a username and password, the variables APPROPEDIA_{NAME,PASSWORD} are used.")
a.add_argument ('--name', help = 'Appropedia login name')
a.add_argument ('--password', help = 'Appropedia password')
a.add_argument ('--description', help = 'File description')
a.add_argument ('file', help = 'Files to upload', nargs = '+')
args = a.parse_args ()
name = args.name or os.getenv ('APPROPEDIA_NAME')
password = args.password or os.getenv ('APPROPEDIA_PASSWORD')
description = args.description or os.getenv ('APPROPEDIA_UPLOAD_DESCRIPTION')
assert name and password and description
# }}}
# {{{ Helper functions for file upload.
def encode_multipart_formdata(fields, files, BOUNDARY = None): # {{{
""" Encodes fields and files for uploading.
fields is a sequence of (name, value) elements for regular form fields - or a dictionary.
files is a sequence of (name, filename, value) elements for data to be uploaded as files.
Return (content_type, body) ready for urllib2.Request instance
You can optionally pass in a boundary string to use or we'll let mimetools provide one.
"""
if BOUNDARY is None:
BOUNDARY = '-----' + mimetools.choose_boundary() + '-----'
CRLF = '\r\n'
L = []
if isinstance(fields, dict):
fields = fields.items()
for (key, value) in fields:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name="%s"' % key)
L.append()
L.append(value)
for (key, filename, value) in files:
filetype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename))
if not filetype.startswith('text'):
L.append('Content-Transfer-Encoding: binary')
L.append('Content-Type: %s' % filetype)
L.append('Content-Length: %d' % len(value))
L.append()
L.append(value)
L.append('--' + BOUNDARY + '--')
L.append()
body = CRLF.join(L)
content_type = 'multipart/form-data; boundary=%s' % BOUNDARY # XXX what if no files are encoded
return content_type, body
# }}}
def build_request(theurl, fields, files, txheaders=None): # {{{
"""Given the fields to set and the files to encode it returns a fully formed urllib2.Request object.
You can optionally pass in additional headers to encode into the opject. (Content-type and Content-length will be overridden if they are set).
fields is a sequence of (name, value) elements for regular form fields - or a dictionary.
files is a sequence of (name, filename, value) elements for data to be uploaded as files.
"""
content_type, body = encode_multipart_formdata(fields, files)
if not txheaders:
txheaders = {}
txheaders['Content-type'] = content_type
txheaders['Content-length'] = str(len(body))
return urllib2.Request(theurl, body, txheaders)
# }}}
# }}}
# {{{ Get token and cookie from login page.
page = urllib2.urlopen ('https://www.appropedia.org/w/index.php?title=Special:UserLogin')
cookie = page.info ()['Set-Cookie']
assert cookie.startswith ('appropedia_w1_session=')
assert ';' in cookie
cookie = cookie[:cookie.index (';')]
content = page.read ()
token = re.findall ('name="wpLoginToken" value="(.*?)"', content)
assert len (token) == 1
token = token[0]
# }}}
# {{{ Use token and log in.
data = (('wpLoginAttempt', 'Log in'), ('wpLoginToken', token), ('wpName', name), ('wpPassword', password))
login = urllib2.urlopen (urllib2.Request ('https://www.appropedia.org/w/index.php?title=Special:UserLogin&action=submitlogin&type=login', urllib.urlencode (data), {'Cookie': cookie}))
# We don't actually use the login page.
# }}}
# {{{ Send files.
for file in args.file:
# {{{ Get token from upload page.
page = urllib2.urlopen (urllib2.Request ('https://www.appropedia.org/Special:Upload', None, {'Cookie': cookie}))
content = page.read ()
token = re.findall ('id="wpEditToken" type="hidden" value="(.*?)"', content)
assert len (token) == 1
token = token[0]
# }}}
# {{{ Build data.
filecontent = open (file, 'rb').read ()
filename = re.findall ('/?([^/]*$)', file)[0]
data = (
('wpDestFile', filename),
('wpUploadDescription', description),
('wpLicense', 'GFDL'),
('wpEditToken', token),
('title', 'Special:Upload'),
('wpDestFileWarningAck', '1'),
('wpUpload', 'Upload file'))
# }}}
# {{{ Send file.
page = urllib2.urlopen (build_request ('https://www.appropedia.org/Special:Upload', data, (('wpUploadFile', filename, filecontent),), {'Cookie': cookie}))
# }}}
# }}}