• Stars
    star
    162
  • Rank 232,284 (Top 5 %)
  • Language
    Crystal
  • License
    MIT License
  • Created almost 7 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Selenium alternative for Crystal. Browser manipulation without the Java overhead.

Marionette

Looking for a co-maintainer for Marionette. If interested just respond in the issue titled "Looking for a co-maintainer".

GitHub Workflow Status License Crystal Version

Marionette is a one-size-fits-all approach to WebDriver adapters. It works with most all web driver implementations, including:

  • Chrome
  • Chromium
  • Firefox
  • Safari
  • Edge
  • Internet Explorer
  • Opera
  • PhantomJS
  • Webkit GTK
  • WPE Webkit
  • Android

Table of Contents

Installation

  1. Make sure you have Crystal installed. This is a Crystal project and Crystal is required for usage. If you don't have it installed, see https://crystal-lang.org.

  2. Add Marionette to an existing project by adding the dependency to your shard.yml

    dependencies:
      marionette:
        github: watzon/marionette
        branch: master
  3. Run shards install to download and install Marionette as a dependency.

  4. Download and have installed at least one WebDriver. See the #webdriver section below for links to various downloads.

WebDriver

WebDriver is a protocol which allows browser implementations to be remote controlled via a common interface. It's because of this functionality that frameworks like Marionette are possible. To use the protocol you first have to have installed one of the many WebDriver implementations, here are some of those:

Firefox

GeckoDriver is implemented and supported by Mozilla directly.

Chrome

ChromeDriver is implemented and supported by the Chromium Project.

Opera

OperaChromiumDriver is implemented and supported by Opera Software.

Safari

SafariDriver is implemented and supported directy by Apple. It comes pre-installed with Safari and Safari Technology Preview.

Edge

Microsoft is implementing and maintaining the Microsoft Edge WebDriver.

Internet Explorer

Only version 11 is supported, and it requires additional configuration.

Note: Marionette specific configuration instructions coming soon.

Getting Started

The goal of Marionette is simplicity, which is why it's written in Crystal. Once you have a webdriver installed and sitting comfortably on your path, using it couldn't be easier:

require "marionette"

session = Marionette::WebDriver.create_session(:chrome)

# Navigate to crystal-lang.org
session.navigate("https://crystal-lang.org")

# Start an action chain and perform it
session.perform_actions do
  # Click the "INSTALL" link
  click ".main-actions a:nth-child(1)"
end

sleep 5
session.close

Driver Capabilities

Different drivers have different capabilities available to them. To make setting them a little easier, there's the DriverOptions module which is extended by Marionette itself. Take, for instance, Chrome:

# Make this instance headless
options = Marionette.chrome_options(args: ["headless"])

# Create a Chrome session
session = Marionette::WebDriver.create_session(:chrome, capabilities: options)

args in this case are arguments to be passed to the browser itself rather than the driver. If you wish to pass arguments to the driver you can use the args parameter in the create_session method.

Browser Manipulation

As shown above, you can initialize a new driver session for whatever driver you want using Marionette::WebDriver.create_session, the first and most important argument to which is :browser. Browser can be any of :chrome, :firefox, :opera, :safari, :edge, :internet_explorer, :webkit_gtk, :wpe_webkit, or :android.

If the driver for the chosen browser is installed under its usual name that should be all you need to do, if not you may need to provide the binary location via the :exe_path argument. Other notable arguments are:

  • :port - sets the port you want the driver to listen on
  • :env - a hash of environment variables for the driver to be aware of
  • :args - a string array of arguments to pass to the webdriver process
  • :options - a JSON compatible structure containing browser options. see here for some nice helpers.

Navigation

#navigate

The first thing you will want to do after launching a browser is to open your website. This can be achieved in a single line:

session.navigate("https://crystal-lang.org")

#current_url

You can read the current URL from the browser’s address bar using:

session.current_url
# => https://crystal-lang.org

#back

Pressing the browser’s back button:

session.back

#forward

Pressing the browser’s forward button:

session.forward

#refresh

Refresh the current page:

session.refresh

#title

You can read the current page title from the browser:

session.title
# => Crystal | The Crystal Programming Language

Windows and Tabs

WebDriver does not make the distinction between windows and tabs. If your site opens a new tab or window, Marionette will let you work with it using a window handle. Each window has a unique identifier which remains persistent in a single session.

#current_window

You can get the currently active window using:

session.current_window

This returns a Window instance containing a handle and allowing certain functions to be performed directly on the window instance.

#windows

You can get an array of all currently opened windows using:

session.windows

#new_window

You can create a new window or tab using:

session.new_window(:window) # default
session.new_window(:tab)

# Or using the Window object

Marionette::Window.new(:window) # default
Marionette::Window.new(:tab)

#switch_to_window

To interact with other windows you have to switch to them, this can be done with:

session.switch_to_window(window)

# Or using the Window object

window.switch

#close_window

When you are finished with a window or tab and it is not the last window or tab open in your browser, you should close it and switch back to the window you were using previously:

session.close_window(window)

# Or using the Window object

window.close

#close_current_window

Think of this is a shortcut to #close_window but for the currently active window:

session.close_current_window

#stop

When you are finished with the browser session you should call stop, instead of close:

session.stop

Stop will:

  • Close all the windows and tabs associated with that WebDriver session
  • Close the browser process
  • Close the background driver process

Stop will be automatically closed on process exit.

Frames and IFrames

Frames are a now deprecated means of building a site layout from multiple documents on the same domain. You are unlikely to work with them unless you are working with an pre HTML5 webapp. Iframes allow the insertion of a document from an entirely different domain, and are still commonly used.

If you need to work with frames or iframes, WebDriver allows you to work with them in the same way. Consider a button within an iframe. If we inspect the element using the browser development tools, we might see the following:

<div id="modal">
  <iframe id="buttonframe" name="myframe"  src="https://watzon.github.io">
   <button>Click here</button>
 </iframe>
</div>

If it was not for the iframe we would expect to click on the button using something like:

session.find_element!("button").click

However, if there are no buttons outside of the iframe, you might instead get a no such element error. This happens because Marionette is only aware of the elements in the top level document. To interact with the button, we will need to first switch to the frame, in a similar way to how we switch windows. WebDriver offers three ways of switching to a frame.

#switch_to_frame

The switch_to_frame session method allows us to tell the WebDriver that we want to switch the page context to the given frame/iframe:

# Find the element
iframe = session.find_element!("#modal>iframe")

# Switch to the frame
session.switch_to_frame(iframe)

# Now we can click the button
session.find_element!("button").click

#switch_to_parent_frame

If you're in a nested set of frames you can switch back to the parent frame using:

session.switch_to_parent_frame

#leave_frame

When you're done inside a frame and want to get back to the normal document context you can use:

session.leave_frame

Window Management

Screen resolution can impact how your web application renders, so WebDriver provides mechanisms for moving and resizing the browser window.

#size

You can fetch window dimensions either collectively or individually:

# First get the current window
window = session.current_window

# Access dimensions individually
width = window.width
height = window.height

# Or collectively as a Size object
window.size
# => Size(width: 800.0, height: 600.0)

#resize_to

To resize the window:

window.resize_to(600, 800)

# Or using an existing Size object `size`

window.size = size

#position

You can fetch the coordinates of the top left corner of the browser window:

window.position
# => Location(x: 0.0, y: 0.0)

#move_to

You can also easily set the position of the window:

window.move_to(250, 250)

# Or using a Location object `location`

window.position = location

#maximize

To maximize the given window:

window.maximize

#minimize

To minimize the given window:

window.minimize

#fullscreen

To make the given window full screen:

window.fullscreen

Working with Elements

Element represents a DOM element. WebElements can be found by searching from the document root using a WebDriver instance, or by searching under another Element.

WebDriver API provides built-in methods to find the WebElements which are based on different properties like ID, Name, Class, XPath, CSS Selectors, link Text, etc.

#active_element

Get and return the active element:

session.active_element

#find_element

find_element and find_element! are used to find and store references to page elements. The only difference between the two is that find_element! will throw an exception if the element is not found and will not be nil, whereas find_element will return nil if the element is not found.

# Get the search box element by name
search_bar = session.find_element!("q", :name)

# Peform an action
search_bar.send_keys("Crystal programming language")

# Click the submit button (using the default :css option this time)
session.find_element!("button.submit").click

#find_elements

Used to find an array of elements matching the given selector:

# Get all elements with the tag name "p"
elements = session.find_elements("p", :tag_name)

elements.each do |el|
  puts el.text
end

#find_child

Used to find a child element within the context of parent element. Like find_element it has raising and non-raising varients:

# Find the search form
form = session.find_element!("form", :tag_name)

# Find the form's child search box
search_bar = form.find_child!("q", :name)

search_bar.send_keys("Crystal programming language")

#find_children

Just like find_elements and find_child:

element = session.find_element!("div", :tag_name)

paragraphs = element.find_children("p", :tag_name)

paragraphs.each do |para|
  puts para.text
end

Contributing

  1. Fork it ( https://github.com/watzon/marionette/fork )
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

This project exists thanks to all the people who contribute.

More Repositories

1

fbmdob

Facebook image Metadata Obfuscation server
Vue
159
star
2

wsl-proxy

WSL proxy files for editor/linux interop
Batchfile
140
star
3

arachnid

Powerful web scraping framework for Crystal
Crystal
78
star
4

salesforce-angular2-boilerplate

Boilerplate for building an Angular 2 application on the Salesforce platform. Includes the ability to develop locally using the Salesforce REST API
TypeScript
33
star
5

JackOS

i686 kernel written in Nim
Nim
29
star
6

adonis-jsonable

AdonisJs Trait Provider that aims to solve the problems with using Postgres' JSON types
JavaScript
26
star
7

cru

LibUI based GUI framework for Crystal
Crystal
25
star
8

ngrok.cr

Ngrok wrapper for Crystal
Crystal
25
star
9

ROT26

Pure Crystal implementation of the ROT26 encryption algorithm
Crystal
23
star
10

telethon-session-mongo

MongoDB backend for Telethon session storage
Python
21
star
11

pixie

Making magic with Crystal and images (using ImageMagick)
Crystal
18
star
12

arg_parser

A powerful JSON::Serializable like argument parser for Crystal
Crystal
18
star
13

zhtml

HTML parser built in Zig
Zig
17
star
14

subnet

Crystal library for working with IPv4 and IPv6 addresses
Crystal
15
star
15

browser

Browser detection library for Crystal
Crystal
13
star
16

xmler

Python library to convert dictionaries into valid XML. Supports namespaces.
Python
12
star
17

nacl

Crystal bindings to libsodium (WIP)
Crystal
12
star
18

octokit.cr

Crystal toolkit for the GitHub API (in development)
Crystal
12
star
19

json-to-crystal

Convert JSON structures into Crystal classes with JSON mappings
JavaScript
12
star
20

paste69

Simple CURL-able pastebin
Crystal
11
star
21

lucky_have_i_been_pwned_validator

Have I Been Pwned password validator for LuckyFramework
Crystal
10
star
22

safety

Safe types for V
V
10
star
23

github-api-nim

Nim wrapper for the GitHub API [WIP]
Nim
10
star
24

spotlight

Search engine parsing for Crystal
Crystal
10
star
25

apatite

Apatite is a fundamental package for scientific computing with Crystal
Crystal
9
star
26

identicon

Pure Crystal identicon generator
Crystal
8
star
27

inkify

API based replacement for carbon.now.sh
Rust
8
star
28

diff

Pure Crystal implementation of various diffing algorithms
Crystal
8
star
29

docker-api

Crystal wrapper for the Docker API
Crystal
8
star
30

opengraph.cr

Crystal wrapper for the Open Graph protocol
Crystal
7
star
31

lucky_inertia

Inertia.js adapter for the Lucky web framework
Crystal
7
star
32

freetype.cr

Crystal bindings to Freetype2
Crystal
7
star
33

vnntp

Implementation of RFC 3977 (Network News Transfer Protocol) for V
V
7
star
34

cor

Make working with colors in Crystal fun!
Crystal
7
star
35

crodoc

Robust Crystal documentation generator inspired by YARD. Just an idea right now, don't get your hopes up.
Crystal
7
star
36

gravity

WIP Annotation based ORM for Crystal
Crystal
6
star
37

robots.cr

Simple robots.txt parser for Crystal
Crystal
6
star
38

fenneco

Crystal
6
star
39

remind_me_bot

Crystal
5
star
40

ffmpeg.cr

Crystal
5
star
41

wasm-crypto

Various Rust crypto libraries wrapped in WASM for use in the browser
TypeScript
5
star
42

telegram.cr

Telegram Bot API library written in Crystal and designed with Lucky integration in mind
Crystal
5
star
43

stringscan.js

StringScanner for JavaScript
JavaScript
4
star
44

lucky_htmx

Crystal
4
star
45

tomi

Unified crypto currency wallet generation API for Crystal
Crystal
4
star
46

massclone

Easily clone all of your gihub repos
Nim
4
star
47

kutt

Simple Crystal API wrapper for kutt.it
Crystal
3
star
48

html_parser

Port of fb55/htmlparser2 for Deno
TypeScript
3
star
49

nimtgbot-starter

C
3
star
50

fckeverywordbot

Inspired by the Fuck Every Word project on Twitter, this is a Telegram bot that does the same
Crystal
3
star
51

easy_oauth

Crystal OAuth headers made simple
Crystal
3
star
52

cracker.js

Easy to use bot/userbot framework built on top of GramJS; Gram + Cracker = 🧡
TypeScript
3
star
53

sterile

Crystal
3
star
54

extensions

Helpful extensions to existing Crystal classes and modules
Crystal
3
star
55

esto.cr

E621 API client for Crystal
Crystal
3
star
56

pluscode

Crystal implementation of Open Location Codes
Crystal
3
star
57

paste69-old

Crystal
3
star
58

coinpayments.cr

CoinPayments API wrapper for Crystal
Crystal
3
star
59

mint-tabler

Tabler icon components for Mint Lang
Mint
3
star
60

sysinfo.cr

Psutil for Crystal
Crystal
3
star
61

crego

Steganography library for Crystal
Crystal
3
star
62

qrv

Generate qr codes with vlang using libqrencode
Coq
3
star
63

kantek

Pluggable Telegram userbot
Python
3
star
64

vento

Telegram Bot API library for V
V
2
star
65

BigInteger.ts

Deno first BigInt wrapper based on peterolson/BigInteger.js
TypeScript
2
star
66

paste69-fresh

TypeScript
2
star
67

utilibot

Crystal
2
star
68

hikoki

Hikōki is a telegram userbot to help me with group and spam management
Python
2
star
69

crystalexbot

Crystal
2
star
70

manycoin.js

An expressive, promise driven wrapper around the JSON-RPC API's for various crypto currencies
JavaScript
2
star
71

waves.cr

WIP WAVE format reader in Crystal
Crystal
2
star
72

tweeter

Twitter API wrapper for Crystal
Crystal
2
star
73

tidal.cr

Unofficial Tidal API client for Crystal
Crystal
1
star
74

synaptic.cr

Architecture-free neural network library for Crystal
Crystal
1
star
75

tgp

WIP Python implementation of kitty's terminal graphics protocol
Python
1
star
76

sirbansabot

Crystal
1
star
77

zedcoinbot

JavaScript
1
star
78

watzon

1
star
79

watzon.tech-contents

1
star
80

structure

Python struct like library for Crystal
Crystal
1
star
81

twitter.cr

Twitter API library
Crystal
1
star
82

sysutil_gem

Ruby
1
star
83

crystal-posh

Poshmark API client for Crystal
Crystal
1
star
84

monocypherjs

Zig
1
star
85

protokit

MTProto + Typescript = ❤️
1
star
86

salesforce_client_tracker

Chrome extension for tracking current clients on salesforce
JavaScript
1
star
87

belljs

GramJS for Deno
TypeScript
1
star
88

crystal-razer

Facilitate control of your Razer chroma devices. Based on python razer-drivers (but much faster).
Crystal
1
star
89

ohshit

Oh Shit! is the command fixer you didn't know you needed. Inspired by thefuck.
Crystal
1
star
90

oxford-dictionary.cr

Oxford Dictionary API wrapper for Crystal
Crystal
1
star
91

fenneco-js

TypeScript
1
star
92

wallet-watcher

Watches your garlicoin wallet and notifies you if there's a change in your balance
Crystal
1
star
93

dolly

WIP port of Playwright for Crystal
Crystal
1
star
94

tgcmndr

Telegram group management and stats
1
star
95

reader

A set of methods for processing keyboard input in character, line and multiline modes.
1
star
96

crystal-emoji-regex

💎 A set of Crystal regular expressions for matching Unicode Emoji symbols.
Crystal
1
star
97

Dell-Precision-5540-Sonoma

Hackintosh configuration for the Dell Precision 5540
C++
1
star
98

aoc2023

CSS
1
star
99

dogefaucetbot

JavaScript
1
star
100

wrapi

REST API wrapping framework written in Crystal
Crystal
1
star