You nailed it when you said "examples, tutorials....". I, too, think that Kevin's suggestion of MechanicalSoup bears looking into - I completely forgot about John's mention of it.
Git still confounds me but I see MechanicalSoup is installable via pip and I've done so.
On another note... every time I run pip (actually, pip3), I get this:
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
I've tried various incantations of that suggestion to no avail. Is pip (pip3), on the i, upgradeable in this fashion or is it via PTF?
Aaaargghh... RoboBrowser - Another option to consider! Jeez, I wish the site I'm using had a web service.
COMMON Certified Application Developer - ILE RPG on IBM i on Power
From: OpenSource <opensource-bounces@xxxxxxxxxxxx> on behalf of John Yeung <gallium.arsenide@xxxxxxxxx>
Sent: Monday, October 2, 2017 3:34 PM
To: IBMi Open Source Roundtable
Subject: Re: [IBMiOSS] Headless Browsing
On Mon, Oct 2, 2017 at 4:58 PM, Pete Helgren <pete@xxxxxxxxxx> wrote:
Since we are all straying outside the Python fold, I use JSoup to accomplish
something similar in Java.....similar I guess to "Beautiful Soup"...
Well, the purpose of straying outside Python has mainly been to say
"here's something that worked for me; you can probably try the
equivalent thing in Python". Jack's response was a little more
mysterious; he just tossed in a link for Selenium, which OP already
mentioned he had looked into.
You are correct that jsoup is pretty much Java's BeautifulSoup, but
BeautifulSoup came first (by about 5 years, and very likely was a
strong influence on the development of jsoup), and it was the other
thing OP already mentioned besides Selenium.
So far Kevin's response has probably been the most helpful. There's no
way he could have known I already recommended MechanicalSoup to OP a
few months ago, when he approached me about this topic off-list.
I think what Roger is mainly missing right now is examples, tutorials,
and specific tips that come from experience, particularly from those
who are familiar with any IBM-i-specific wrinkles in the endeavor.
Unfortunately, I only have a tiny bit of experience with Selenium, via
Python on Windows. I never got to the point where I felt like I really
knew what I was doing (and a few months later, when I had some time to
try to expand on my early experiments, I discovered the site I was
trying to interact with changed some stuff which broke what I'd done
earlier). But a lot of that is just me not knowing much about Web
stuff in general.
I will say I agree with Kevin that if you can achieve what you need
without a full-blown browser (which is what Selenium is), it's usually
preferable to do so. Of course, you can't always tell what you can
achieve with something until you try it, so there's not really much
for it except to try it. I would try MechanicalSoup or RoboBrowser
first. (Whichever one seems friendlier or more intuitive.)
Roger, have you already attempted anything? If so, where are you getting stuck?
As an Amazon Associate we earn from qualifying purchases.