diff --git a/README.md b/README.md index 8929b22..2b5898b 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,42 @@ # Lightning Network Daemon Developer Site -Documentation for the Lightning Network Daemon +Developer guides and resources for the Lightning Network Daemon + +## Overview + +This repository contains functionality for programmatically pulling guides from +the lnd repo, using a Python script and Jinja2 templates to output markdown +files, which are the corresponding pages in the fully rendered Jekyll static +site. + +Pay special attention to these files: +- `templates/base.md`: The Jinja2 template fed into the Python + script, holding the Jeykll header to be prepended to the guide content pulled + from Github. The `templates` dir also holds the templates for special cases + where for example a dev site-specific footer needs to be appended +- `update_and_render.sh`: Update local guides to the latest version available +- `render.py`: The Python script that uses local guides and Jinja template to + strip out redundant titles, add page headers/footers, and output Jekyll + markdown +- `deploy.sh`: Build static site from Jekyll markdown and deploy to Google Cloud + Platform + +The rest of the files in this repo are the standard Jekyll site files. ## Running the site locally +### Prerequisites + +You're going to need: + + - **Linux or OS X** — Windows may work, but is unsupported. + - **Ruby, version 2.2.5 or newer** + - **Bundler** — If Ruby is already installed, but the `bundle` command doesn't work, just run `gem install bundler` in a terminal. + +### Running locally + +```shell +git clone https://github.com/lightninglabs/lightning-dev-site +``` Install Jekyll: ``` $ gem install jekyll bundler @@ -11,23 +45,65 @@ Run the site and watch for changes: ``` $ bundle exec jekyll serve ``` +* If running remotely add: `--host=0.0.0.0` -## Deployment -The Lightning Dev Site is deployed with `s3_website`. Visit their [github -repo](https://github.com/laurilehmijoki/s3_website) for more information. +## Regenerating documentation -### Steps +```shell +# Install Jinja for python templating +pip install Jinja2 +``` + +### Get the latest INSTALL.md +```shell +curl -o INSTALL.md -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/docs/INSTALL.md +``` + +### Fetch the latest Docker guide +```shell +curl -o DOCKER-README.md -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/docker/README.md +``` -1. Add the deployment credentials for `s3_config.yml` +### Get the latest gRPC guides +```shell +curl -o python.md -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/docs/grpc/python.md +curl -o javascript.md -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/docs/grpc/javascript.md ``` -$ export LN_S3_ID="YOUR_S3_ID" -$ export LN_S3_SECRET="YOUR_S3_SECRET" -$ export LN_CLOUDFRONT_DISTRIBUTION_ID="YOUR_CLOUDFRONT_DISTRIBUTION_ID" + +Let's run the script to render the guides: +```shell +python render.py ``` -2. Deploy the site from local changes: +Now that you're all set up, you can just run `./update_and_render.sh` to +automatically pull the latest markdown files and render the local Jekyll docs. + +## Deployment + +The Lightning API is deployed with Google Cloud Platform. Visit [this blog +post](https://little418.com/2015/07/jekyll-google-cloud-storage.html) for more +information. +### Steps + +1. Install Google Cloud SDK and authenticate into it: +```bash +brew cask install google-cloud-sdk +gcloud auth login +``` + +2. Build +```bash +bundle exec jekyll build ``` -s3_website push + +3. Push to Google Cloud Bucket +```bash +# -m use faster multithreaded uploads +# -d delete remote files that aren't in the source +# -r recurse into source subdirectories +gsutil -m rsync -d -r ./_site gs://dev.lightning.community ``` + +In the future, you can just run `./deploy.sh` to deploy automatically. diff --git a/_config.yml b/_config.yml index e283820..a90951d 100644 --- a/_config.yml +++ b/_config.yml @@ -14,14 +14,13 @@ # You can create any custom variable you would like, and they will be accessible # in the templates via {{ site.myvariable }}. title: Lightning Network Developers -email: maxfangx@gmail.com +email: hello@lightning.engineering description: > # this means to ignore newlines until "baseurl:" - Developer resources, documentation, and education for the Lightning Network - Daemon. + Developer resources and documentation for the Lightning Network Daemon. baseurl: "" # the subpath of your site, e.g. /blog -url: "" # the base hostname & protocol for your site, e.g. http://example.com +url: "http://dev.lightning.community" # the base hostname & protocol for your site, e.g. http://example.com twitter_username: lightning -github_username: lightninglabs +github_username: lightningnetwork/lnd # Build settings markdown: kramdown @@ -34,11 +33,15 @@ permalink: pretty # Exclude from processing. # The following items will not be processed, by default. Create a custom list # to override the default setting. -# exclude: -# - Gemfile -# - Gemfile.lock -# - node_modules -# - vendor/bundle/ -# - vendor/cache/ -# - vendor/gems/ -# - vendor/ruby/ +exclude: + - Gemfile + - Gemfile.lock + - node_modules + - vendor/bundle/ + - vendor/cache/ + - vendor/gems/ + - vendor/ruby/ + - templates + - render.py + - update_and_render.sh + - s3_website.yml diff --git a/_includes/footer.html b/_includes/footer.html new file mode 100644 index 0000000..744eb45 --- /dev/null +++ b/_includes/footer.html @@ -0,0 +1,49 @@ + diff --git a/_includes/head.html b/_includes/head.html new file mode 100644 index 0000000..3ebfe3c --- /dev/null +++ b/_includes/head.html @@ -0,0 +1,17 @@ + + + + + + {% if page.title %}{{ page.title | escape }}{% else %}{{ site.title | escape }}{% endif %} + + + + + + + + {% if jekyll.environment == 'production' and site.google_analytics %} + {% include google-analytics.html %} + {% endif %} + diff --git a/_includes/header.html b/_includes/header.html index cd7f645..ffcf620 100644 --- a/_includes/header.html +++ b/_includes/header.html @@ -4,7 +4,7 @@ {% assign default_paths = site.pages | map: "path" %} {% assign page_paths = site.header_pages | default: default_paths %} {{ site.title | escape }} - + {% if page_paths %} {% endif %} diff --git a/_includes/icon-github.html b/_includes/icon-github.html new file mode 100644 index 0000000..cc6e48a --- /dev/null +++ b/_includes/icon-github.html @@ -0,0 +1 @@ +{% include icon-github.svg %}lnd diff --git a/_layouts/home.html b/_layouts/home.html index 132bc42..9aad10d 100644 --- a/_layouts/home.html +++ b/_layouts/home.html @@ -4,15 +4,92 @@
-

Building on the Lightning Network Daemon

-

New to lnd? - Installation and - Tutorial +

LND Developer Site

+ +
+

Resources and documentation for the + Lightning Network Daemon (LND) +

+
+ +
+

Learn

+

New to LND? + Overview, + Installation, and + Tutorial +

+

Curious about the Lightning Network? Resources

+
+ +
+

Develop

+

Looking for detailed dev manuals? Developer Guides

+

API Reference Documentation? API site

+

Want to help build the future? Contribute

+

Looking to read the code? Github

+
+ +
+

Community

+

Follow the + Lightning Labs community blog + to keep posted about LND.

-

Need a reference? Documentation

-

Want to learn about Lightning Network? Resources

-

Want to help build the future? Contribute

- - {{ content }} +

Join the + LND Developer Slack + to meet the team, discuss development, and hang out! +

+
+ +
+

Ecosystem

+

See the cool apps and wallets built on Lightning: Lapp directory

+
+ +
+

Overview

+

+ The Lightning Network scales blockchains and enables trustless instant + payments by keeping most transactions off-chain and leveraging the + security of the underlying blockchain as an arbitration layer. +

+

+ This is accomplished primarily through "payment-channels", wherein two + parties commit funds and pay each other by updating the balance + redeemable by either party in the channel. This process is instant and + saves users from having to wait for block confirmations before they can + render goods or services. +

+

+ Payment channels are trustless, since any attempt to defraud the current + agreed-upon balance in the channel results in the complete forfeiture of + funds by the liable party. +

+

+ By moving payments off-chain, the cost of opening and closing channels + (in the form of on-chain transaction fees) is ammortized over the volume + of payments in that channel, enabling micropayments and small-value + transactions for which the on-chain transaction fees would otherwise be + too expensive to justify. Furthermore, the Lightning Network scales not + with the transaction throughput of the underlying blockchain, but with + modern data processing and latency limits - payments can be made nearly + as quickly as packets can be sent. +

+

+ Hash Time-Locked Contracts (HTLCs) allow transactions to be sent between + parties who do not have a direct channels by routing it through multiple + hops, so anyone connected to the Lightning Network is part of a single, + interconnected global financial system. +

+

+ In short, the Lightning Network enables scalable blockchains through a + high-volume of instant transactions not requiring custodial delegation. +

+

+ Continue reading at the full LND Overview and + Developer Guide. +

+
diff --git a/_pages/contribute.md b/_pages/contribute.md index 9ad9c72..9eac39e 100644 --- a/_pages/contribute.md +++ b/_pages/contribute.md @@ -13,8 +13,10 @@ permalink: /contribute/ 4.2. [Testing](#Testing)
4.3. [Code Documentation and Commenting](#CodeDocumentation)
4.4. [Model Git Commit Messages](#ModelGitCommitMessages)
-4.5. [Code Spacing](#CodeSpacing)
-4.6. [Protobuf Compilation](#Protobuf)
+4.5. [Ideal Git Commit Structure](#IdealGitCommitStructure)
+4.6. [Code Spacing](#CodeSpacing)
+4.7. [Protobuf Compilation](#Protobuf)
+4.8. [Additional Style Constraints On Top of gofmt](ExtraGoFmtStyle)
5. [Code Approval Process](#CodeApproval)
5.1. [Code Review](#CodeReview)
5.2. [Rework Code (if needed)](#CodeRework)
@@ -44,8 +46,8 @@ represent real money and introducing bugs and security vulnerabilities can have far more dire consequences than in typical projects where having a small bug is minimal by comparison. In the world of cryptocurrencies, even the smallest bug in the wrong area can cost people a significant amount of money. For this -reason, the Lightning Network Daemon (lnd) has a formalized and rigorous -development process (heavily insipred by +reason, the Lightning Network Daemon (`lnd`) has a formalized and rigorous +development process (heavily inspired by [btcsuite](https://github.com/btcsuite)) which is outlined on this page. We highly encourage code contributions, however it is imperative that you adhere @@ -84,7 +86,7 @@ security and performance implications. ### 3. Required Reading -- [Effective Go](http://golang.org/doc/effective_go.html) - The entire lnd +- [Effective Go](http://golang.org/doc/effective_go.html) - The entire `lnd` project follows the guidelines in this document. For your code to be accepted, it must follow the guidelines therein. - [Original Satoshi Whitepaper](https://bitcoin.org/bitcoin.pdf) - This is the white paper that started it all. Having a solid @@ -100,10 +102,10 @@ Note that the core design of the Lightning Network has shifted over time as concrete implementation and design has expanded our knowledge beyond the original white paper. Therefore, specific information outlined in the resources above may be a bit out of date. Many implementers are currently working on an -initial [Version 1 Specification](https://medium.com/@lightningnetwork/lightning-network-meeting-on-interoperability-and-specifications-ea49e47696a4). +initial [Lightning Network Specifications](https://github.com/lightningnetwork/lightning-rfc). Once the specification is finalized, it will be the most up-to-date comprehensive document explaining the Lightning Network. As a result, it will -be recommened for newcomers to read first in order to get up to speed. +be recommended for newcomers to read first in order to get up to speed. @@ -138,7 +140,7 @@ This approach has several benefits: #### 4.2. Testing -One of the major design goals of all of lnd's packages and the daemon itself is +One of the major design goals of all of `lnd`'s packages and the daemon itself is to aim for a high degree of test coverage. This is financial software so bugs and regressions in the core logic can cost people real money. For this reason every effort must be taken to ensure the code is as accurate and bug-free as @@ -166,9 +168,13 @@ A quick summary of test practices follows: be accompanied by unit tests exercising the new or changed behavior. - Changes to behavior within the daemon's interaction with the P2P protocol, or RPC's will need to be accompanied by integration tests which use the - [`networkHarness`framework](https://github.com/lightningnetwork/lnd/blob/master/networktest.go) + [`networkHarness`framework](https://github.com/lightningnetwork/lnd/blob/master/lntest/harness.go) contained within `lnd`. For example integration tests, see - [`lnd_test.go`](https://github.com/lightningnetwork/lnd/blob/master/lnd_test.go#L181). + [`lnd_test.go`](https://github.com/lightningnetwork/lnd/blob/master/lntest/itest/lnd_test.go#L203). + +Throughout the process of contributing to `lnd`, you'll likely also be +extensively using the commands within our `Makefile`. As a result, we recommend +[perusing the make file documentation](https://github.com/lightningnetwork/lnd/blob/master/docs/MAKEFILE.md). @@ -277,7 +283,7 @@ Further paragraphs come after blank lines. Here are some of the reasons why wrapping your commit messages to 72 columns is a good thing. -- git log doesn’t do any special wrapping of the commit messages. With +- git log doesn't do any special wrapping of the commit messages. With the default pager of less -S, this means your paragraphs flow far off the edge of the screen, making them difficult to read. On an 80 column terminal, if we subtract 4 columns for the indent on the left and 4 more for symmetry on the @@ -292,19 +298,39 @@ all short-[commit messages are to be prefixed according to the convention outlined in the Go project](https://golang.org/doc/contribute.html#change). All commits should begin with the subsystem or package primarily affected by the change. In the case of a widespread change, the packages are to be delimited by -either a '+' or a ','. This prefix seems minor but can be extremly helpful in +either a '+' or a ','. This prefix seems minor but can be extremely helpful in determining the scope of a commit at a glance, or when bug hunting to find a commit which introduced a bug or regression. + + +#### 4.5. Ideal Git Commit Structure + +Within the project we prefer small, contained commits for a pull request over a +single giant commit that touches several files/packages. Ideal commits build on +their own, in order to facilitate easy usage of tools like `git bisect` to `git +cherry-pick`. It's preferred that commits contain an isolated change in a +single package. In this case, the commit header message should begin with the +prefix of the modified package. For example, if a commit was made to modify the +`lnwallet` package, it should start with `lnwallet: `. + +In the case of changes that only build in tandem with changes made in other +packages, it is permitted for a single commit to be made which contains several +prefixes such as: `lnwallet+htlcswitch`. This prefix structure along with the +requirement for atomic contained commits (when possible) make things like +scanning the set of commits and debugging easier. In the case of a change that +touches several packages, and can only compile with the change across several +packages, a `multi: ` prefix should be used. + -#### 4.5. Code Spacing +#### 4.6. Code Spacing -Blocks of code within lnd should be segmented into logical stanzas of +Blocks of code within `lnd` should be segmented into logical stanzas of operation. Such spacing makes the code easier to follow at a skim, and reduces unnecessary line noise. Coupled with the commenting scheme specified above, proper spacing allows readers to quickly scan code, extracting semantics quickly. -Functions should _not_ just be layed out as a bare contiguous block of code. +Functions should _not_ just be laid out as a bare contiguous block of code. **WRONG** ```go @@ -348,15 +374,23 @@ Functions should _not_ just be layed out as a bare contiguous block of code. -#### 4.5.6. Protobuf Compilation +#### 4.7. Protobuf Compilation -The `lnd` project uses `protobuf`, and its extension [`gRPC`](www.grpc.io) in +The `lnd` project uses `protobuf`, and its extension [`gRPC`](https://www.grpc.io/) in several areas and as the primary RPC interface. In order to ensure uniformity of all protos checked, in we require that all contributors pin against the _exact same_ version of `protoc`. As of the writing of this article, the `lnd` -project uses [v3.2.0](https://github.com/google/protobuf/releases/tag/v3.2.0) +project uses [v3.4.0](https://github.com/google/protobuf/releases/tag/v3.4.0) of `protoc`. +The following commit hashes of related projects are also required in order to +generate identical compiled protos and related files: + * grpc-ecosystem/grpc-gateway: `f2862b476edcef83412c7af8687c9cd8e4097c0f` + * golang/protobuf: `ab9f9a6dab164b7d1246e0e688b0ab7b94d8553e` + +For detailed instructions on how to compile modifications to `lnd`'s `protobuf` +definitions, check out the [lnrpc README](https://github.com/lightningnetwork/lnd/blob/master/lnrpc/README.md). + Additionally, in order to maintain a uniform display of the RPC responses rendered by `lncli`, all added or modified `protof` definitions, _must_ attach the proper `json_name` option for all fields. An example of such an option can @@ -373,12 +407,40 @@ Notice how the `json_name` field option corresponds with the name of the field itself, and uses a `snake_case` style of name formatting. All added or modified `proto` fields should adhere to the format above. + + +#### 4.8. Additional Style Constraints On Top of `gofmt` + +Before a PR is submitted, the proposer should ensure that the file passes the +set of linting scripts run by `make lint`. These include `gofmt`. In addition +to `gofmt` we've opted to enforce the following style guidelines. + + * ALL columns (on a best effort basis) should be wrapped to 80 line columns. + Editors should be set to treat a tab as 4 spaces. + * When wrapping a line that contains a function call as the unwrapped line + exceeds the column limit, the close paren should be placed on its own + line. Additionally, all arguments should begin in a new line after the + open paren. + + **WRONG** + ```go + value, err := bar(a + a, b, c) + ``` + + **RIGHT** + ```go + value, err := bar( + a, a, b, c, + ) + ``` + ### 5. Code Approval Process This section describes the code approval process that is used for code -contributions. This is how to get your changes into lnd. +contributions. This is how to get your changes into `lnd`. @@ -427,17 +489,27 @@ master. In certain cases the code reviewer(s) or interested committers may help you rework the code, but generally you will simply be given feedback for you to make the necessary changes. +During the process of responding to review comments, we prefer that changes be +made with [fixup commits](https://thoughtbot.com/blog/autosquashing-git-commits). +The reason for this is two fold: it makes it easier for the reviewer to see +what changes have been made between versions (since Github doesn't easily show +prior versions like Critique) and it makes it easier on the PR author as they +can set it to auto squash the fix up commits on rebase. + This process will continue until the code is finally accepted. #### 5.3. Acceptance -Once your code is accepted, it will be integrated with the master branch. -Typically it will be rebased and fast-forward merged to master as we prefer to -keep a clean commit history over a tangled weave of merge commits. However, -regardless of the specific merge method used, the code will be integrated with -the master branch and the pull request will be closed. +Once your code is accepted, it will be integrated with the master branch. After +2+ (sometimes 1) LGTM's (approvals) are given on a PR, it's eligible to land in +master. At this final phase, it may be necessary to rebase the PR in order to +resolve any conflicts and also squash fix up commits. Ideally, the set of +[commits by new contributors are PGP signed](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work), +although this isn't a strong requirement (but we prefer it!). In order to keep +these signatures intact, we prefer using merge commits. PR proposers can use +`git rebase --signoff` to sign and rebase at the same time as a final step. Rejoice as you will now be listed as a [contributor](https://github.com/lightningnetwork/lnd/graphs/contributors)! @@ -449,7 +521,7 @@ Rejoice as you will now be listed as a [contributor](https://github.com/lightnin #### 6.1. Contribution Checklist -- [  ] All changes are Go version 1.5 compliant +- [  ] All changes are Go version 1.11 compliant - [  ] The code being submitted is commented according to the [Code Documentation and Commenting](#CodeDocumentation) section - [  ] For new code: Code is accompanied by tests which exercise both @@ -459,10 +531,13 @@ Rejoice as you will now be listed as a [contributor](https://github.com/lightnin - [  ] Any new logging statements use an appropriate subsystem and logging level - [  ] Code has been formatted with `go fmt` -- [  ] Running `go test` does not fail any tests +- [  ] For code and documentation: lines are wrapped at 80 characters + (the tab character should be counted as 8 characters, not 4, as some IDEs do + per default) +- [  ] Running `make unit` and `make itest` do not fail any tests - [  ] Running `go vet` does not report any issues -- [  ] Running [golint](https://github.com/golang/lint) does not - report any **new** issues that did not already exist +- [  ] Running `make lint` does not report any **new** issues that + did not already exist diff --git a/_pages/docs.md b/_pages/docs.md deleted file mode 100644 index 477d444..0000000 --- a/_pages/docs.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -layout: page -title: Docs -permalink: /docs/ ---- - -This is the base Jekyll theme. You can find out more info about customizing your Jekyll theme, as well as basic Jekyll usage documentation at [jekyllrb.com](https://jekyllrb.com/) - -You can find the source code for Minima at GitHub: -[jekyll][jekyll-organization] / -[minima](https://github.com/jekyll/minima) - -You can find the source code for Jekyll at GitHub: -[jekyll][jekyll-organization] / -[jekyll](https://github.com/jekyll/jekyll) - - -[jekyll-organization]: https://github.com/jekyll diff --git a/_pages/installation.md b/_pages/installation.md deleted file mode 100644 index 9f3eef8..0000000 --- a/_pages/installation.md +++ /dev/null @@ -1,194 +0,0 @@ ---- -layout: page -title: Installation -permalink: /installation/ ---- - -# Preliminaries - In order to build from source, the following build dependencies are - required: - - * **Go:** Installation instructions can be found [here](http://golang.org/doc/install). - - It is recommended to add `$GOPATH/bin` to your `PATH` at this point. - **Note:** If you are building with `Go 1.5`, then you'll need to - enable the vendor experiment by setting the `GO15VENDOREXPERIMENT` - environment variable to `1`. If you're using `Go 1.6` or later, then - it is safe to skip this step. - - * **Glide:** This project uses `Glide` to manage dependencies as well - as to provide *reproducible builds*. To install `Glide`, execute the - following command (assumes you already have Go properly installed): - ``` - $ go get -u github.com/Masterminds/glide - ``` - -# Installing LND - -With the preliminary steps completed, to install `lnd`, `lncli`, and all -related dependencies run the following commands: -``` -$ git clone https://github.com/lightningnetwork/lnd $GOPATH/src/github.com/lightningnetwork/lnd -$ cd $GOPATH/src/github.com/lightningnetwork/lnd -$ glide install -$ go install . ./cmd/... -``` - -**Updating** - -To update your version of `lnd` to the latest version run the following -commands: -``` -$ cd $GOPATH/src/github.com/lightningnetwork/lnd -$ git pull && glide install -$ go install . ./cmd/... -``` - -**Tests** - -To check that `lnd` was installed properly run the following command: -``` -go install; go test -v -p 1 $(go list ./... | grep -v '/vendor/') -``` - -# Installing BTCD - -`lnd` currently requires `btcd` with segwit support, which is not yet merged -into the master branch. Instead, [roasbeef](https://github.com/roasbeef/btcd) -maintains a fork with his segwit implementation applied. To install, run the -following commands: - -Install **btcutil**: (must be from roasbeef fork, not from btcsuite) -``` -$ go get -u github.com/roasbeef/btcutil -``` - -Install **btcd**: (must be from roasbeef fork, not from btcsuite) -``` -$ cd $GOPATH/src/github.com/roasbeef/btcd -$ glide install -$ go install . ./cmd/... -``` - -## Environment Setup - -# Simnet vs. Testnet Development - -If you are doing local development, such as for the tutorial, you'll want to -start both `btcd` and `lnd` in the `simnet` mode. Simnet is similar to regtest -in that you'll be able to instantly mine blocks as needed to test `lnd` -locally. In order to start either daemon in the `simnet` mode use `simnet` -instead of `testnet`, such as adding the `--bitcoin.simnet` flag instead of the -`--bitcoin.testnet` flag. - -Another relevant command line flag for local testing of new `lnd` developments -is the `--debughtlc` flag. When starting `lnd` with this flag, it'll be able to -automatically settle a special type of HTLC sent to it. This means that you -won't need to manually insert invoices in order to test payment connectivity. -To send this "special" HTLC type, include the `--debugsend` command at the end -of your `sendpayment` commands. -``` -$ lnd --bitcoin.active --bitcoin.simnet --debughtlc -``` - -# Starting btcd - -Running the following command will create `rpc.cert` and default `btcd.conf` -(replace `--testnet` with `--simnet` if you want to run on simnet) - -``` -$ btcd --testnet --txindex --rpcuser=kek --rpcpass=kek -``` -If you want to use `lnd` on testnet, `btcd` needs to first fully sync the -testnet blockchain. Depending on your hardware, this may take up to a few -hours. - -(NOTE: It may take several minutes to find segwit-enabled peers.) - -While `btcd` is syncing you can check on its progress using btcd's `getinfo` -RPC command: -``` -$ btcctl --testnet getinfo -{ - "version": 120000, - "protocolversion": 70002, - "blocks": 1114996, - "timeoffset": 0, - "connections": 7, - "proxy": "", - "difficulty": 422570.58270815, - "testnet": true, - "relayfee": 0.00001, - "errors": "" -} -``` - -Additionally, you can monitor btcd's logs to track its syncing progress in real -time. - -You can test your `btcd` node's connectivity using the `getpeerinfo` command: -``` -$ btcctl --testnet getpeerinfo | more -``` - -# Starting lnd - -If you are on testnet, run this command after `btcd` has finished syncing. -Otherwise, replace `--bitcoin.testnet` with `--bitcoin.simnet` -``` -$ lnd --bitcoin.active --bitcoin.testnet --debuglevel=debug --externalip=X.X.X.X -``` - -If you'd like to signal to other nodes on the network that you'll accept -incoming channels (as peers need to connect inbound to initiate a channel -funding workflow), then the `--externalip` flag should be set to your publicly -reachable IP address. - -# Creating an lnd.conf (Optional) - -Optionally, if you'd like to have a persistent configuration between `lnd` -launches, allowing you to simply type `lnd --bitcoin.testnet --bitcoin.active` -at the command line, you can create an `lnd.conf`. - -**On MacOS, located at:** -`/Users/[username]/Library/Application Support/Lnd/lnd.conf` - -**On Linux, located at:** -`~/.lnd/lnd.conf` - -Here's a sample `lnd.conf` to get you started: -``` -[Application Options] -debuglevel=trace -debughtlc=true -maxpendingchannels=10 -profile=5060 -externalip=128.111.13.23 -externalip=111.32.29.29 - -[Bitcoin] -bitcoin.active -bitcoin.rpchost=localhost:18334 -``` - -Notice the `[Bitcoin]` section. This section houses the parameters for the -Bitcoin chain. Also `lnd` also supports Litecoin, one is able to also specified -(but not concurrently with Bitcoin!) the proper parameters, so `lnd` knows to -be active on Litecoin's testnet4. - -# Accurate as of: -- _roasbeef/btcd commit:_ `54362e17a5b80643ef1e12edc08895a2e2a1202b` -- _roasbeef/btcutil commit:_ `d347e49` -- _lightningnetwork/lnd commit:_ `d7b36c6` - -## Next Steps - -* **[Tutorial](/tutorial/):** Get acquainted with the skills necessary for `lnd` development. -* **[Step-by-step send payment guide with - docker](/docker-guide/):** This - guide describes how to package `lnd` with `btcd` together to make deployment - easier. -* **[Resources](/resources/):** Learn more about the Lightning Network -* **[Reference Documentation](/docs/):** Read the docs. -* **[Code Contribution Guidelines](/contribute/):** Contribute to `lnd` itself. - diff --git a/_pages/lapps.md b/_pages/lapps.md new file mode 100644 index 0000000..96358b5 --- /dev/null +++ b/_pages/lapps.md @@ -0,0 +1,168 @@ +--- +layout: page +title: Lightning App Directory +permalink: /lapps/ +--- + +### Introduction + +This page lists wallets and Lightning apps ("lapps") built on LND, along with several +compatible apps built on Eclair and c-lightning. + +### Wallets + +* [Lightning App](https://github.com/lightninglabs/lightning-app) by Lightning Labs: + a cross-platform wallet powered by `lnd`. See the announcement for + [desktop](https://blog.lightning.engineering/announcement/2019/04/23mainnet-app.html) + and [mobile](https://blog.lightning.engineering/announcement/2019/06/19/mobile-app.html) + for more information. Checkout the [code](https://github.com/lightninglabs/lightning-app) + and find the releases for + [desktop](https://github.com/lightninglabs/lightning-app/releases), + [Android](https://play.google.com/apps/testing/engineering.lightning.LightningMainnet) + and [iOS](https://testflight.apple.com/join/xx23MrBp). +Screenshot of Lightning Desktop App + +* [Zap](https://zap.jackmallers.com/) Wallet by Jack Mallers: a mobile and desktop app focused on ease of + use and general user experience. Check out the [Zap website](https://zap.jackmallers.com/) or read the code on + [GitHub](https://github.com/LN-Zap). +Zap Wallet payment request view + +* [HTLC.me](https://htlc.me/): Hosted wallet great for quickly trying out + Lightning payments on the Bitcoin testnet. +* [ZeusLN](https://zeusln.app/): + A mobile Bitcoin app for Lightning Network Daemon (lnd) node operators. Runs on Android and iOS. +* [Breez](https://breez.technology/): Breez is a Lightning Network mobile client and a hub. + It provides a platform for simple, instantaneous bitcoin payments. Supports Android and iOS. + +### Integrations + +* [Bitrefill](https://en.bitrefill.com/): Recharge prepaid phone cards with + Bitcoin and Litecoin +* [Y'alls](http://yalls.org/): Read and write articles, with Lightning Network + micropayments +Screenshot of Yalls home view +* [LNCast](http://lncast.com/): Lightning Network Podcasts +* [Bard](https://www.bard.fun/): Simple paywall for watching a music visualizer +* [HodlHodl.com](https://hodlhodl.com/?set_asset=btcln): Peer-to-peer cryptocurrency + exchange with Lightning integration +* [Zigzag.io](https://zigzag.io/#/): Cryptocurrency trading using Lightning + (custodial) +* [lntxbot](https://telegram.me/lntxbot) Lightning integration to Telegram with + tipping, send and receive and satellite API usage (custodial) +* [paywall.link](https://paywall.link/) Put any website behind a paywall and pay with Lightning + +### Lightning network explorers + +* [1ml.com](https://1ml.com/) +* [explore.casa](https://explore.casa/) +* [explorer.acinq.co](https://explorer.acinq.co/) +* [ln.alhur.es](https://ln.alhur.es/) + +### Protocol Services + +* [Lightning Faucet](https://faucet.lightning.community/): Receive free testnet + Bitcoin +* [lnd.fun](http://lnd.fun/): Panel for webmasters to manage their full lightning node +Screenshot of lnd.fun dashboard home view + +### Developer Tools + +* [WooCommerce + Plugin](https://github.com/joaodealmeida/woocommerce-gateway-lightning): + Gateway plugin to accept Lightning payments at WooCommerce stores +* [LND Explorer](https://demo1.lndexplorer.com/): demo for a web interface for + LND. Code on [Github](https://github.com/altangent/lnd-explorer) +* [Light-weight LND Dashboard](https://github.com/mably/lncli-web): A + lightweight web client for LND +* [LightningJ](http://www.lightningj.org/): A project intending to simplify the + integration of Lightning implementations for Java developers, containing + simple to use API implementations and converters between JSON and XML. + +### Tipping +* [LightningTip](https://github.com/michael1011/lightningtip): Library to accept + tips via the Lightning Network +* [Slack tipbot](https://github.com/CryptoFR/ln-tip-slack): Custodial Slack + tipbot +* [LightningPay-PHP](https://github.com/robclark56/lightningPay-PHP): PHP plugin + for eCommerce website to add _Pay by Lightning_ to accept order payments via the + Lightning Network. +* [Tippin.me](https://tippin.me/) LN tips integrated to Twitter (custodial) +* [Lightning Gifts](https://lightning.gifts/) Create fee-less Bitcoin gift + vouchers to share with friends, family, and your haters. Powered by the Lightning Network ⚡ + +### Gaming +* [Lightning Gem](https://lightninggem.com/): Betting game using Lightning for + payments +* [Kriptode.com](https://kriptode.com/) Lightning based games and more. +* [Satoshis.place](https://satoshis.place/) Internet graffiti paid with Lightning + +### Physical applications +* [Ben Arc](https://twitter.com/BTCSocialist)'s various [projects](https://github.com/arcbtc/) +based on Arduino, ESP32 and the M5Stack. + * [M5StackSats](https://github.com/arcbtc/M5StackSats): + A bitcoin point of sale terminal using the ESP32 based M5Stack + * [Physically faucet](https://github.com/arcbtc/physically-faucet): + A Lightning Network bitcoin faucet using the api + * [Flux capacitor](https://github.com/arcbtc/flux-capacitor): + A dynamic Lightning Network point of sale device + * [The Fossa](https://github.com/arcbtc/The-Fossa): + The Free and Open Source Software (FOSS) ATM, dump that stinky loose fiat change for glorious sats! +* [LightningATM](https://github.com/21isenough/LightningATM): + This ATM accepts coins and sends bitcoin over the Lightning Network + +### Eclair Lapps + +Eclair is a Scala implementation of the Lightning Network built by [ACINQ](https://acinq.co/) + +* [Eclair](https://github.com/ACINQ/eclair) app: on + [desktop](https://github.com/ACINQ/eclair/releases) and + [Android](https://play.google.com/store/apps/details?id=fr.acinq.eclair.wallet.mainnet2) +* [Starblocks](https://starblocks.acinq.co/#/): Virtual coffee shop +* [Strike](https://strike.acinq.co/#/): Stripe-like Lightning payment aggregator + API (custodial) +* [Lightning Conductor](http://lightningconductor.net/): A service for + converting Lightning balances to BTC and back without having to close + or open channels, currently on testnet. + +### c-Lightning Lapps + +c-lightning is a specification-compliant LN implementation in C, under the +[Elements Project](https://elementsproject.org/) + +* [Blockstream store](https://store.blockstream.com/) (mainnet): Bitcoin + paraphernalia that can only be purchased over LN. +* [Lightning Charge](https://github.com/ElementsProject/lightning-charge): A + drop-in solution for accepting lightning payments +* [WooCommerce + Plugin](https://github.com/ElementsProject/woocommerce-gateway-lightning): + Original c-lightning version of the Gateway plugin to accept Lightning + payments at WooCommerce stores, based on Lightning Charge. +* [Week of + Lapps](https://blockstream.com/2018/03/29/blockstreams-week-of-lapps-ends.html) + built on Lightning Charge by Nadav Idgi. + * [Nanopos](https://github.com/ElementsProject/nanopos): A simple + point-of-sale system for fixed-price goods + * [FileBazaar](https://github.com/ElementsProject/filebazaar): A system for + selling files such as documents, images, and videos + * [Lightning Publisher for + WordPress](https://github.com/ElementsProject/wordpress-lightning-publisher): + A patronage model for unlocking WordPress blog entries + * [Paypercall](https://github.com/ElementsProject/paypercall): A programmer’s + toolkit for Lightning that enables micropayments for individual API calls + * [Ifpaytt](https://github.com/ElementsProject/ifpaytt): An extension of + paypercall that allows web developers using IFTTT to request payments for + service usage + * [Lightning Jukebox](https://github.com/ElementsProject/lightning-jukebox): + A fun demo that reimagines a classic technology for the Lightning Network + * [Nanotip](https://github.com/ElementsProject/nanotip): The simple tip jar, + rebuilt to issue Lightning Network invoices +* [Elaine Ou's projects](https://elaineou.com/shop/): + * [Lightning-powered vending machine](https://github.com/elaineo/Jellybean) + * [Bitcoin-payable Twitter bot with Lightning Charge](https://github.com/elaineo/lightningbot) + * [Twitter relay for Lightning JSON-RPC interface](https://github.com/elaineo/LightningBuddy) + +### Requests + +If you would like a Lightning app considered for this page, please send a +link and a one sentence description to . +If applicable, please specify which Lightning implementation it is built on. diff --git a/_pages/overview.md b/_pages/overview.md new file mode 100644 index 0000000..206a4b2 --- /dev/null +++ b/_pages/overview.md @@ -0,0 +1,410 @@ +--- +layout: page +title: LND Overview and Developer Guide +permalink: /overview/ +--- + +### Introduction + +The LND Overview and Developer Guide aims to provide *just* enough information +about LND to enable readers to build applications. It starts with a conceptual +review of the Lightning Network, before jumping into the important aspects of +working specifically with LND. If you are already comfortable with how the +Lightning Network works, feel free to skip down to the [Components](#components) +section. The command line examples are for illustrative purposes only and will +be covered in more depth in the [installation guide](/guides/installation/) and +[tutorial](/tutorial/). + +#### Preliminaries + +This overview assumes basic knowledge of Bitcoin mechanics. If terms like "UTXO" +and "locktime" are unfamiliar to you, you should refer to the [Bitcoin developer +guide](https://bitcoin.org/en/developer-guide), which serves a similar purpose. + +### Table of Contents + * [Lightning Network](#lightning-network) + * [Payment Channels](#payment-channels) + * [Channel Updates](#channel-updates) + * [Multihop payments](#multihop-payments) + * [Network Topology](#network-topology) + * [Integration Guidelines](#integration-guidelines) + * [Components](#components) + * [Network Layers](#network-layers) + * [Software Components](#software-components) + * [LND Interfaces](#lnd-interfaces) + * [Channel Lifecycle](#channel-lifecycle) + * [Payment Lifecycle](#payment-lifecycle) + * [Payment Requests](#payment-requests) + * [Payment flow](#payment-flow) + * [Conclusion](#conclusion) + +![Lightning Network Graph](http://imgur.com/xqfllBI.png) + +### Lightning Network + +The Lightning Network scales blockchains and enables trustless instant payments +by keeping most transactions off-chain and leveraging the security of the +underlying blockchain as an arbitration layer. + +This is accomplished primarily through "payment-channels", wherein two parties +commit funds and pay each other by updating the balance redeemable by either +party in the channel. This process is instant and saves users from having to +wait for block confirmations before they can render goods or services. + +Payment channels are trustless, since any attempt to defraud the current +agreed-upon balance in the channel results in the complete forfeiture of funds +by the liable party. + +By moving payments off-chain, the cost of opening and closing channels (in the +form of on-chain transaction fees) is ammortized over the volume of payments in +that channel, enabling micropayments and small-value transactions for which the +on-chain transaction fees would otherwise be too expensive to justify. +Furthermore, the Lightning Network scales not with the transaction throughput of +the underlying blockchain, but with modern data processing and latency limits - +payments can be made nearly as quickly as packets can be sent. + +Hash Time-Locked Contracts (HTLCs) allow transactions to be sent between parties +who do not have a direct channel by routing them through multiple hops, so anyone +connected to the Lightning Network is part of a single, interconnected global +financial system. + +In short, the Lightning Network enables scalable blockchains through a +high-volume of instant transactions not requiring custodial delegation. + +### Payment Channels + +Payment channels are the main workhorse of the Lightning Network. They allow +multiple transactions to be aggregated into just a few on-chain transactions. + +In the vast majority of cases, someone only needs to broadcast the first and +last transaction in the channel. +- The Funding Transaction creates the channel. During this stage, funds + are sent into a multisig address controlled by both Alice and Bob, the + counterparties to the channel. This address can be funded as a single-payer + channel or by both Alice and Bob. +- The Closing Transaction closes the channel. When broadcast, the multisig + address spends the funds back to Alice and Bob according to their agreed-upon + channel amount. + +![Commitment transaction Alice 5BTC Bob 5BTC](http://imgur.com/rqHWEoC.png) + +In the case where either party attempts to defraud the other, a third +transaction, which punishes the attacker, will end up being broadcasted +on-chain. Let's investigate how this is possible by the way Lightning does +channel updates. + +#### Channel Updates + +In between the opening and closing transactions broadcast to the blockchain, +Alice and Bob can create a near infinite number of intermediate closing +transactions that gives different amounts to the two parties. + +For example, if the initial state of the channel credits both Alice and Bob with +5BTC out of the 10BTC total contained in the multisig address, Alice can make a +1BTC payment to Bob by updating the closing transaction to pay 4BTC/6BTC, where +Alice is credited with 4BTC and Bob with 6BTC. Alice will give the signed +transaction to Bob, which is equivalent to payment, because Bob can broadcast it +at any time to claim his portion of the funds. Similarly, Alice is also able to +broadcast the closing transaction at any time to claim her funds. + +![Channel Update Alice 4BTC Bob 6BTC](http://imgur.com/auACasH.png) + +To prevent an attack where Alice voids her payment by broadcasting the initial +state of 5BTC/5BTC, there needs to be a way to revoke prior closing +transactions. Payment revocation roughly works like the following. + +When Alice broadcasts a closing transaction to the blockchain, she is attesting +to the current state of the chain. But since there may be millions of closing +transactions in a channel, all of which are valid, the blockchain itself can't +tell if what Alice attested to was indeed the correct state. Therefore, Alice +must wait 3 days after broadcasting the closing transaction before she can +redeem her funds. During this time, Bob is given a chance to reveal a secret +that will allow him to sweep Alice's funds immediately. Alice can thus revoke +her claim to the money in some state by giving Bob the secret to the closing +transaction. This allows Bob to take all of Alice's money, but only if Alice +attest to this old state by broadcasting the corresponding closing transaction +to the blockchain. + +Channel updates are thus fully trustless. When making an update, both parties +exchange the secrets for the prior state, so that all prior states will have +been revoked except for the current state. Both parties will never broadcast an +old state, because they know the other party can take all their money if they +do so. + +### Multihop payments + +Single channels work well if you have a financial relationship with some entity +where you make payments frequently or in metered amounts. But most payments, +like purchasing an umbrella from a corner store because you lost it again, are +one-off. For Lightning to help Bitcoin scale for general use cases, there needs +to be a way for the whole network to forward payments through channels that +already exist. Furthermore, this process should retain the trustless nature of +individual channels, otherwise it becomes too hard to identify dishonest actors +amongst a large number of hops. + +Once you don't have to trust the intermediaries, you no longer even care who +they are. This allows Lightning nodes to be fully anonymous, which is a huge win +for privacy. + +Concretely, Suppose Alice has a channel with Bob, who has a channel with Carol, +who has a channel with Dave: `A<->B<->C<->D`. How can Alice pay Dave? + +Alice first notifies Dave that she wants to send him some money. + +In order for Dave to accept this payment, he must generate a random number `R`. +He keeps `R` secret, but hashes it and gives the hash `H` to Alice. + +![Dave gives hash H to Alice](http://imgur.com/sXuL8Tn.png) + +Alice tells Bob: "I will pay you if you can produce the preimage of `H` within 3 +days." In particular, she signs a transaction where for the first three days +after it is broadcast, only Bob can redeem it with knowledge of R, and +afterwards it is redeemable only by Alice. This transaction is called a Hash +Time-Locked Contract (HTLC) and allows Alice to make a conditional promise to +Bob while ensuring that her funds will not be accidentally burned if Bob never +learns what R is. She gives this signed transaction to Bob, but neither of them +broadcast it, because they are expecting to clear it out later. + +![Alice creates HTLC with Bob](http://imgur.com/aNQoA9Z.png) + +Bob, knowing that he can pull funds from Alice if he knows R, now has no issue +telling Carol: "I will pay you if you can produce the preimage of H within *2* +days." + +Carol does the same, making an HTLC that will pay Dave if Dave can produce R +within 1 day. However, Dave does in fact know R. Because Dave is able to pull +the desired amount from Carol, Dave can consider the payment from Alice +completed. Now, he has no problem telling R to Carol and Bob so that they are +able to collect their funds as well. + +![Dave distributes R](http://imgur.com/nTLWBbm.png) + +Now, everyone can clear out, because they have a guaranteed way to pull their +deserved funds by broadcasting these HTLCs onto Bitcoin's network (i.e. +on-chain). They would prefer not to do that though, since broadcasting on-chain +is more expensive, and instead settle each of these hops off chain. Alice knows +that Bob can pull funds from her since he has `R`, so she tells Bob: "I'll pay +you, regardless of `R`, and in doing so we'll terminate the HTLC so we can +forget about R." Bob does the same with Carol, and Carol with Dave. + +![Everyone terminates their HTLCs](http://imgur.com/iRx4bf5.png) + +Now, what if Dave is uncooperative and refuses to give `R` to Bob and Carol? +Note that Dave must broadcast the transaction from Carol within 1 day, and in +doing so must reveal R in order to redeem the funds. Bob and Carol can simply +look at the blockchain to determine what R is and settle off-chain as well. + +We have shown how to make a payment across the Lightning Network using only +off-chain transactions, without requiring direct channel links or trusting any +intermediaries. As long as there is a path from the payer to the payee, payments +can be routed, just like the Internet. + +### Network Topology + +The expected shape / network topology of the Lightning Network will depend on +behavior implemented in the varying Lightning implementations as well as actual +usage. Users do not have to manually manage their channels, since `lnd` has an +'autopilot' feature including settings optimizing for both everyday use and fee +revenue. And of course, channels can be opened on demand via the standard +command line, gRPC, and REST interfaces. + +### Integration guidelines + +When integrating `lnd`, hot and cold storage must be considered. To maximize +security, we generally want to keep as little as possible in hot wallets, and as +much as possible in cold wallets. + +It is possible to construct Lightning channels where the keys are cold, but they +would need to be brought back online to conduct a channel update. Only with hot +wallets can the Lightning Network attain a high volume of transactions. + +This is only a surface level introduction to Lightning integration. For a more +illustrative example of how Lightning Network may work in production, check out +the "Integration Components" and "Security Considerations" sections of the +[Exchange Integration Document](https://docs.google.com/document/d/1r38-_IgtfOkhJh4QbN7l6bl7Rol05qS-i7BjM3AjKOQ/edit) +maintained by Bryan Vu. + +### Components + +#### Network Layers + +The Lightning Network is an *overlay* network on top of another blockchain. To +avoid confusion it is crucial to differentiate between the following network +layers we encounter when reasoning about `lnd`: +- Bitcoin/Litecoin Network: This is the underlying blockchain that `lnd` rests + on top of. `lnd` needs a way to communicate with the underlying blockchain in + order to send on-chain payments, create channel open/close transactions, and + watch for events on the blockchain. +- P2P Network: This is the peer layer where `lnd` nodes add each other as peers + so they can send messages between one another via an [encrypted + connection](https://github.com/lightningnetwork/lightning-rfc/blob/master/08-transport.md). + For example, the `lncli connect` adds a peer, which are identified by identity + pubkey and IP address. +- Payment channel network: This is the layer where nodes are connected by + payment channels. For example, the `lncli openchannel` command opens a + channel with a node that was already connected at the peer layer, and the + `lncli describegraph` command returns the list of edges and vertices of the + payment channel graph. + +#### Software Components + +There are distinct software components we should be aware of when developing on +- [`btcd`](https://github.com/btcsuite/btcd) or [`bitcoind`](https://github.com/bitcoin/bitcoin) is used by `lnd` to interface with the underlying blockchain. +- `lnd` / `lncli`: LND stands for Lightning Network Daemon and serves as the + main software component driving the Lightning Network. It manages a database, + connects to peers, opens / closes channels, generates payment invoices, sends, + forwards, and revokes payments, responds to potential breaches, and more. + `lncli` opens up a command line interface for driving `lnd`. +- [Neutrino](https://github.com/lightninglabs/neutrino) is an experimental + Bitcoin light client designed to support Lightning mobile clients. This is a + wallet UI usable with `lnd`. Neutrino is not required from an application + development standpoint, but can be regarded as the primary way for an + end-user of `lnd` to interact with the Bitcoin Network and the applications + built on top of it. + +#### LND Interfaces + +There are several ways to drive `lnd`. + +- `lncli` is the `lnd` command line tool. All commands are executed + instantaneously. A full list of commands can be viewed with `lncli --help`. + To see a breakdown of the parameters for a particular command, run + `lncli --help` +- gRPC is the preferred programmatic way to interact with `lnd`. It includes simple + methods that return a response immediately, as well as response-streaming and + bidrectional streaming methods. Check out the guides for working with gRPC for + [Python](/guides/python-grpc/) and [Javascript](/guides/javascript-grpc/) +- LND also features a REST proxy someone can use if they are accustomed to + standard RESTful APIs. However, REST does not have full streaming RPC coverage. + +All of these LND interfaces are documented in the [API +Reference](//api.lightning.community), featuring a description of the +parameters, responses, and code examples for Python, Javascript, and command +line arguments if those exist. + +### Channel Lifecycle + +To better understand the development workflow around Lightning channels, it is +worthwhile to examine step by step the lifecycle of a payment channel. It +contains roughly 4 steps: + +1. **Adding a peer.** Before a channel can be opened between two Lightning + nodes, they must first be able to securely communicate with each other. This + is accomplished with the `ConnectPeer` RPC method or `connect` on `lncli`. + ```shell + lncli connect @: + ``` +2. **Initiating Channel Opening.** The `OpenChannel` method begins the channel + opening process with a connected peer. Lightning assumes that this is led by + a single party. The opening party can specify a local amount, representing + the funds they would like to commit to the channel, and a "push" amount, the + amount of money that they would like to give to the other side as part of an + initial commitment state. One could imagine that instead of sending a + standard Bitcoin transaction to pay a merchant, they could instead open a + channel with the push amount representing the amount they want to pay, and + optionally add some funds of their own, so that both parties can benefit from + having a channel available for payments in the future. + ```shell + lncli openchannel --node_key= --local_amt= + ``` + Lightning Wallet App Open Channel +3. **Wait for confirmations.** To prevent double spending attacks on the channel + opening transaction, users should specify the `--block` `lncli` command line + argument. So after initializing the channel opening process, it is often + required to mine a few blocks: + ```shell + btcctl generate 6 + ``` +4. **Close Channel.** If either party in a channel no longer wants to keep it + open, they can close it at any time with the `CloseChannel` method. + ```shell + lncli closechannel --funding_txid= --output_index= + ``` + +### Payment Lifecycle + +Because Lightning payments are instant, its API tends to be much simpler, since +there is no need to wait for block confirmations before a payment is considered +accepted. It resembles a fairly standard payment flow, but there are a few +additional things to keep in mind. + +#### Payment Requests + +Payment requests, often also referred to as Invoices, are a simple, extensible +protocol compatible with QR-codes. It includes a 6-character checksum in case +there is a mistake with copy/paste or manual entry. + +Payment requests are composed of two sections: +1. **Human readable part:** Contains a prefix `ln` followed by an optional + amount. +2. **Data part**: Contains a UTC Unix timestamp and optionally some tagged + parts, as well as a signature over the human readable and data parts. + - Tagged parts include a payment hash, the pubkey of the payee node, a + description of the purpose of payment, an expiration time (default to 1 + hour if not specified), and extra routing information. Some tagged parts + are required and others are not. + +Because the payment request contains the payment hash, **payment requests must +be strictly single use**. After an invoice is fulfilled, the hash preimage +becomes publically known. An attacker could save the preimages they've seen and +reuse it for another payment that is reusing the invoice. Therefore, **failure +to generate new payment requests means that an on-path attacker can steal the +payment en route.** + +Another detail worth noting is that payees should not accept payments after the +payment request has expired (`timestamp` + `expiry`), and payers likewise should +not attempt them. This will affect any web application with `lnd` integration, +since if an invoice for a good or service is not fulfilled within the given +timeframe, a new one should be generated. + +Other possibly unexpected rules include that the payee should accept up to +twice the amount encoded in the transaction, so that the payer can make payments +harder to track by adding in small variations. + +A full specification of the payment request data format, required and optional +parts, and required behavior can be found in [BOLT +11](https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md). + +#### Payment flow + +Let's now see what an ideal payment flow looks like. + +1. **Create Invoice:** The recipient creates an invoice with a specified value, + expiration time, and an optional memo. If there was already an invoice + created for this good and it expired, or a sufficient amount of time has + elapsed, a fresh invoice should be generated. + ```shell + lncli addinvoice --amt=6969 --memo="A coffee for Roger" + ``` + Lightning Wallet Generate Payment Request +2. **Check invoice:** The payer decodes the invoice to see the destination, + amount and payment hash. This way, they can validate that the invoice was + legitimate, and that they aren't being defrauded or overcharged. At this + stage, the user should also check that the expiration time of the invoice has + not passed + ```shell + lncli decodepayreq --pay_req= + ``` +3. **Send payment:** The payer sends their payment, possibly routed through the + Lightning Network. Developers can do this through an LND interface and end + users can use the desktop or mobile app. + ```shell + lncli sendpayment --pay_req= + ``` + Lightning Wallet send payment screen +4. **Check payment:** The recipient checks that their invoice has been + fulfilled. They make a call to the `LookupInvoice` command, which returns + this information in the `settled` field. + ```shell + lncli lookupinvoice --rhash= + ``` + Lightning Wallet 5BTC Received + +We have now covered the basic workflow for generating invoices and +sending/receiving payments. + +### Conclusion + +You have completed the conceptual overview of LND and a high level primer on the +components and workflows. To get started on developing, check out the +[installation guide](/guides/installation/) and [tutorial](/tutorial/). diff --git a/_pages/resources.md b/_pages/resources.md index a662515..30b345e 100644 --- a/_pages/resources.md +++ b/_pages/resources.md @@ -1,18 +1,171 @@ --- layout: page -title: Resources +title: Lightning Resources permalink: /resources/ --- -This is the base Jekyll theme. You can find out more info about customizing your Jekyll theme, as well as basic Jekyll usage documentation at [jekyllrb.com](https://jekyllrb.com/) +### Overview -You can find the source code for Minima at GitHub: -[jekyll][jekyll-organization] / -[minima](https://github.com/jekyll/minima) +This page is for those seeking to develop on LND or enhance their conceptual +understanding of the Lightning Network and associated technologies. It is not +comprehensive, but a careful study of the materials below should be sufficient +for anyone's needs. -You can find the source code for Jekyll at GitHub: -[jekyll][jekyll-organization] / -[jekyll](https://github.com/jekyll/jekyll) +### Lightning: Beginner +* [What is the Lightning Network and how can it help Bitcoin scale?](https://coincenter.org/entry/what-is-the-lightning-network) + article by Elizabeth Stark + * Accessible article explanation of the Lightning Network by Elizabeth Stark, + the CEO of Lightning Labs. -[jekyll-organization]: https://github.com/jekyll +* [The Blockchain and Us: Interview with Elizabeth Stark](https://www.youtube.com/watch?v=a3HulqfzyYE) + * Quick 7 minute primer on the Lightning Network and its implications. + +* [Joseph Poon discusses Lightning + at Construct 2017](http://www.coindesk.com/events/construct-2017/videos/) + (ctrl+f for "Lightning") + * In-depth spoken explanation of the Lightning Network for the general + audience. Note that this video requires providing CoinDesk an email address. + +* ["Understanding the Lightning + Network"](https://bitcoinmagazine.com/articles/understanding-the-lightning-network-part-building-a-bidirectional-payment-channel-1464710791/): + Aaron van Wirdum's General Explainer Series + * In-depth written explanation great for non-developers. + * [Part 1](https://bitcoinmagazine.com/articles/understanding-the-lightning-network-part-building-a-bidirectional-payment-channel-1464710791/): + Building a Bidirectional Bitcoin Payment Channel + * [Part 2](https://bitcoinmagazine.com/articles/understanding-the-lightning-network-part-creating-the-network-1465326903/): + Creating the Network + * [Part 3](https://bitcoinmagazine.com/articles/understanding-the-lightning-network-part-completing-the-puzzle-and-closing-the-channel-1466178980/): + Completing the Puzzle and Closing the Channel + +* [#bitcoin-lightning - Things to Know](https://medium.com/@rusty_lightning/bitcoin-lightning-things-to-know-e5ea8d84369f): + Rusty Russell addresses FAQs about Lightning, including its requirements, + limitations, segwit, and expected network dynamics. + +### Lightning: Intermediate + +* [Lightning Network Tech + Talk](https://www.youtube.com/watch?v=wIhAmTqXhZQ) at Coinbase January + 2016 + * Spoken in-depth mechanics of bidirectional payment channels and multi-hop + payments + +* [Lightning Networks](https://rusty.ozlabs.org/?p=450): + Rusty Russell's Technical Explainer Series + * In-depth explanation of Lightning, oriented towards developers. + * [Part I](https://rusty.ozlabs.org/?p=450): Revocable Transactions + * [Part II](https://rusty.ozlabs.org/?p=462): Hashed Timelock Contracts (HTLCs) + * [Part III](https://rusty.ozlabs.org/?p=467): Channeling Contracts + * [Part IV](https://rusty.ozlabs.org/?p=477): Summary + +* [Lightning Network Paper](https://lightning.network/lightning-network-paper.pdf) + published February 2015, last updated January 2016 + * The protocol has changed since this original paper, but covers the mid-level + mechanics of the Lightning Network with an emphasis on the smart contracts + that make it trustless + * Combines the HTLC and channel update concepts together + * Explains how the Lightning Network is represented in Bitcoin transaction + based contracts, and how the Lightning Network works with the UTXO model. + +* [Lightning Network Community Blog](http://lightning.community), managed by + Lightning Labs. + * Stay up to date with the latest about LND and the Lightning Network! + +### Lightning: Advanced + +* Read the code on Github: [`lnd`](https://github.com/lightningnetwork/lnd) + +* [Basis Of Lightning + Technology (BOLTs)](https://github.com/lightningnetwork/lightning-rfc/blob/master/00-introduction.md) + * Full technical specification of the Lightning Network. BOLTs are the + common standard enabling distinct, interoperable Lightning implementations. + * [BOLT #0](https://github.com/lightningnetwork/lightning-rfc/blob/master/00-introduction.md): + Introduction and Index + * [BOLT #1](https://github.com/lightningnetwork/lightning-rfc/blob/master/01-messaging.md): + Base Protocol + * [BOLT #2](https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md): + Peer Protocol for Channel Management + * [BOLT #3](https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md): + Bitcoin Transaction and Script Formats + * [BOLT #4](https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md): + Onion Routing Protocol + * [BOLT #5](https://github.com/lightningnetwork/lightning-rfc/blob/master/05-onchain.md): + Recommendations for On-chain Transaction Handling + * [BOLT #6](https://github.com/lightningnetwork/lightning-rfc/blob/master/06-irc-announcements.md): + Interim Node and Channel Discovery + * [BOLT #7](https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md): + P2P Node and Channel Discovery + * [BOLT #8](https://github.com/lightningnetwork/lightning-rfc/blob/master/08-transport.md): + Encrypted and Authenticated Transport + * [BOLT #9](https://github.com/lightningnetwork/lightning-rfc/blob/master/09-features.md): + Assigned Feature Flags + * [BOLT #11](https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md): + Invoice Protocol for Lightning Payments + +* [Lightning Network Deep + Dive](https://www.youtube.com/watch?v=b_szGaaPPFk) talk at SF Bitcoin Devs + August 2016 + * Commitment scheme: Compact revocation derivation and storage + * Revocation scheme: Pre-image derived revocation keys (homomorphic + derivation) + * Lightning Commitment Protocol: the fast link-layer protocol between two + Lightning nodes + * LND architecture + +* [Lightning Network as a Directed Graph: Single-Funded Channel Network + Topology](https://www.youtube.com/watch?v=-lgYYz3y_hY) talk at SF Bitcoin Devs + April 2016 + * Covers single funded channels and channel exhaustion + * Covers how to construct sub-satoshi micropayments: "Pre-Image Length + Probabilistic Payments" + +* [Onion Routing in Lightning](https://youtu.be/Gzg_u9gHc5Q?t=2m47s) - Laolu + Osuntokun at Scaling Bitcoin Milan 2016 (starts at 2:47) + * A way to preserve privacy in the Lightning Network + * [Video](https://youtu.be/Gzg_u9gHc5Q?t=2m47s), + [transcript](https://scalingbitcoin.org/transcript/milan2016/onion-routing-in-lightning) + +* [Outsourced Channel Monitoring](https://youtu.be/Gzg_u9gHc5Q?t=48m12s) - Tadge + Dryja at Scaling Bitcoin Milan 2016 (starts at 48:12) + * Outsourced channel monitoring: delegating the task of channel monitoring to + a semi-trusted peer + * [Video](https://youtu.be/Gzg_u9gHc5Q?t=48m12s), [slides](https://scalingbitcoin.org/milan2016/presentations/D1%20-%208%20-%20Tadge%20Dryja.pdf), [transcript](https://scalingbitcoin.org/transcript/milan2016/unlinkable-outsourced-channel-monitoring) + +### LND Developer Tools + +* [Developer Guides](/guides/) + * Guides walking you through installation, docker setup, Python / Javascript + gRPC, and more +* [API documentation](https://api.lightning.community) + * API Reference documentation for the Lightning Network Daemon +* Lightning Faucet for [Bitcoin testnet](https://faucet.lightning.community/) or + [Litecoin testnet](https://ltc.faucet.lightning.community/) + * To read more, build from source, or deploy your own faucet see the [Github + repository](https://github.com/lightninglabs/lightning-faucet) +* [LND Web Dashboard](https://github.com/mably/lncli-web) + * A web client and dashboard made by Francis Mably. Useful as a more + customizable client or as a starting point for a Lightning project +* [Lightning Apps and the Emerging Developer Ecosystem on LND](http://lightning.community/software/lnd/lightning/2017/07/05/emerging-lightning-developer-ecosystem/) + * A survey of existing apps built on Lightning, as of July 2017. Potentially + useful as inspiration or code examples +* Neutrino: A Bitcoin Light Client used in LND + * [Compact Client Side Filtering for Light Clients](https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-June/014474.html) + on Bitcoin Dev Mailing List + * [Neutrino BIP (Bitcoin Improvement Proposal)](https://github.com/Roasbeef/bips/blob/master/gcs_light_client.mediawiki) + * [Github repository](https://github.com/lightninglabs/neutrino) + +### LND Community + +* LND Developer Slack + * Come here to meet the Lightning community, ask for help, and hang out! + * [invite link](https://join.slack.com/t/lightningcommunity/shared_invite/enQtMzQ0OTQyNjE5NjU1LWRiMGNmOTZiNzU0MTVmYzc1ZGFkZTUyNzUwOGJjMjYwNWRkNWQzZWE3MTkwZjdjZGE5ZGNiNGVkMzI2MDU4ZTE) + +* [#lnd](https://webchat.freenode.net/?channels=lnd) IRC + * [![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)](https://webchat.freenode.net/?channels=lnd) + +* [#lightning-dev](https://webchat.freenode.net/?channels=lightning-dev) IRC + * [![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)](https://webchat.freenode.net/?channels=lightning-dev) + + +If something important is missing, or you feel that additional resources or +links would be helpful, please send a message in the [LND Developer Slack](https://join.slack.com/t/lightningcommunity/shared_invite/enQtMzQ0OTQyNjE5NjU1LWRiMGNmOTZiNzU0MTVmYzc1ZGFkZTUyNzUwOGJjMjYwNWRkNWQzZWE3MTkwZjdjZGE5ZGNiNGVkMzI2MDU4ZTE) diff --git a/_pages/tutorial.md b/_pages/tutorial.md deleted file mode 100644 index d0ab754..0000000 --- a/_pages/tutorial.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -layout: page -title: Tutorial -permalink: /tutorial/ ---- - -# Overview - -This tutorial will get you up to speed with all the skills necessary to work -with `lnd`. We will be building a Bitcoin micropayments paywell into a news -site! - -This tutorial assumes you have completed installation of Go, `btcd`, and `lnd` -on simnet. If not, you can find the installation instructions -[here](/installation/). - -# Stages - -Each of the different stages of this tutorial is oriented towards a specific -skill. Beginners should start at [stage 1](/tutorial/01-lncli). Otherwise, jump -wherever you want below! - -- **[Stage 1 - Setting up a local cluster](/tutorial/01-lncli)** - - How to set up a local environment with three nodes that can make payments - with and through one another. - - Gain experience interacting with `lnd` from the command line. -- **[Stage 2 - Web client](/tutorial/02-web-client)** - - How to run and use a `lnd` web GUI. -- **[Stage 3 - RPC Client](/tutorial/03-rpc-client)** - - How to set up and interact with `lnd` from an RPC client, specifically - `gRPC`. -- **[Stage 4 - Webapp integration](/tutorial/04-rpc-client)** - - Authenticating into a server using `lnd` - - Exposure to `lnrpc` documentation - - Adding a micropayments paywall. - - Recommended next steps diff --git a/_sass/.DS_Store b/_sass/.DS_Store new file mode 100644 index 0000000..04ea361 Binary files /dev/null and b/_sass/.DS_Store differ diff --git a/_sass/minima.scss b/_sass/minima.scss new file mode 100644 index 0000000..4dd099a --- /dev/null +++ b/_sass/minima.scss @@ -0,0 +1,52 @@ +@charset "utf-8"; + +// Define defaults for each variable. + +$base-font-family: "Helvetica Neue", Helvetica, Arial, sans-serif !default; +$base-font-size: 16px !default; +$base-font-weight: 400 !default; +$small-font-size: $base-font-size * 0.875 !default; +$base-line-height: 1.5 !default; + +$spacing-unit: 20px !default; + +$text-color: #111 !default; +$background-color: #fdfdfd !default; +$brand-color: #6a33e2 !default; +$lightning-blue: #3a8aff !default; +$lightning-dkpurple: #17062d !default; +$lightning-white: white !default; + +$grey-color: #828282 !default; +$grey-color-light: lighten($grey-color, 40%) !default; +$grey-color-dark: darken($grey-color, 25%) !default; + +// Width of the content area +$content-width: 800px !default; + +$on-palm: 600px !default; +$on-laptop: 800px !default; + +// Use media queries like this: +// @include media-query($on-palm) { +// .wrapper { +// padding-right: $spacing-unit / 2; +// padding-left: $spacing-unit / 2; +// } +// } +@mixin media-query($device) { + @media screen and (max-width: $device) { + @content; + } +} + +@mixin relative-font-size($ratio) { + font-size: $base-font-size * $ratio; +} + +// Import partials. +@import + "minima/base", + "minima/layout", + "minima/syntax-highlighting" +; diff --git a/_sass/minima/.DS_Store b/_sass/minima/.DS_Store new file mode 100644 index 0000000..5008ddf Binary files /dev/null and b/_sass/minima/.DS_Store differ diff --git a/_sass/minima/_base.scss b/_sass/minima/_base.scss new file mode 100644 index 0000000..24481c8 --- /dev/null +++ b/_sass/minima/_base.scss @@ -0,0 +1,219 @@ +/** + * Reset some basic elements + */ +body, h1, h2, h3, h4, h5, h6, +p, blockquote, pre, hr, +dl, dd, ol, ul, figure { + margin: 0; + padding: 0; +} + + + +/** + * Basic styling + */ +body { + font: $base-font-weight #{$base-font-size}/#{$base-line-height} $base-font-family; + color: $text-color; + background-color: $background-color; + -webkit-text-size-adjust: 100%; + -webkit-font-feature-settings: "kern" 1; + -moz-font-feature-settings: "kern" 1; + -o-font-feature-settings: "kern" 1; + font-feature-settings: "kern" 1; + font-kerning: normal; + + line-height: 23px; /*adjust as needed, slightly higher than default line-height will help with readability */ +} + + + +/** + * Set `margin-bottom` to maintain vertical rhythm + */ +h1, h2, h3, h4, h5, h6, +p, blockquote, pre, +ul, ol, dl, figure, +%vertical-rhythm { + margin-bottom: $spacing-unit; +} + + + +/** + * Images + */ +img { + max-width: 100%; + vertical-align: middle; +} + + + +/** + * Figures + */ +figure > img { + display: block; +} + +figcaption { + font-size: $small-font-size; +} + + + +/** + * Lists + */ +ul, ol { + margin-left: $spacing-unit; +} + +li { + > ul, + > ol { + margin-bottom: 0; + } +} + + + +/** + * Headings + */ +h1, h2, h3, h4, h5, h6 { + font-weight: $base-font-weight; +} + + + +/** + * Links + */ +a { + color: $brand-color; + text-decoration: none; + + /* Blue used for a:visited and text */ + &:visited { + color: $lightning-blue; + } + + &:hover { + // color: $text-color; + // color: darken($brand-color, 15%); + text-decoration: underline; + } + + .social-media-list &:hover { + text-decoration: none; + + .username { + text-decoration: underline; + } + } +} + + +/** + * Blockquotes + */ +blockquote { + color: $grey-color; + border-left: 4px solid $grey-color-light; + padding-left: $spacing-unit / 2; + @include relative-font-size(1.125); + letter-spacing: -1px; + font-style: italic; + + > :last-child { + margin-bottom: 0; + } +} + + + +/** + * Code formatting + */ +pre, +code { + @include relative-font-size(0.9375); + border: 1px solid $grey-color-light; + border-radius: 3px; + background-color: #eef; +} + +code { + padding: 1px 5px; +} + +pre { + padding: 8px 12px; + overflow-x: auto; + + > code { + border: 0; + padding-right: 0; + padding-left: 0; + } +} + + + +/** + * Wrapper + */ +.wrapper { + max-width: -webkit-calc(#{$content-width} - (#{$spacing-unit} * 2)); + max-width: calc(#{$content-width} - (#{$spacing-unit} * 2)); + margin-right: auto; + margin-left: auto; + padding-right: $spacing-unit; + padding-left: $spacing-unit; + @extend %clearfix; + + @include media-query($on-laptop) { + max-width: -webkit-calc(#{$content-width} - (#{$spacing-unit})); + max-width: calc(#{$content-width} - (#{$spacing-unit})); + padding-right: $spacing-unit / 2; + padding-left: $spacing-unit / 2; + } +} + + + +/** + * Clearfix + */ +%clearfix:after { + content: ""; + display: table; + clear: both; +} + + + +/** + * Icons + */ +.icon > svg { + display: inline-block; + vertical-align: middle; + + path { + fill: $grey-color; + } +} + +.social-media-list { + .icon { + padding-right: 5px; + } + + li + li { + padding-top: 5px; + } +} diff --git a/_sass/minima/_layout.scss b/_sass/minima/_layout.scss new file mode 100644 index 0000000..9800520 --- /dev/null +++ b/_sass/minima/_layout.scss @@ -0,0 +1,262 @@ +/** + * Site header + */ +.site-header { + border-top: 5px solid $grey-color-dark; + border-bottom: 1px solid $grey-color-light; + min-height: $spacing-unit * 1.865; + + // Positioning context for the mobile navigation icon + position: relative; + + color: $lightning-white; + background-color: $lightning-dkpurple; +} + +.site-title { + @include relative-font-size(1.625); + font-weight: 300; + + line-height: $base-line-height * $base-font-size * 2.25; + letter-spacing: -1px; + margin-bottom: 0; + float: left; + + &, + &:visited { + // color: $grey-color-dark; + color: $lightning-white; + } +} + +.site-nav { + float: right; + line-height: $base-line-height * $base-font-size * 2.25; + + .nav-trigger { + display: none; + } + + .menu-icon { + display: none; + } + + .page-link { + color: $lightning-white; + // color: $text-color; + + line-height: $base-line-height; + + // Gaps between nav items, but not on the last one + &:not(:last-child) { + margin-right: 20px; + } + } + + @include media-query($on-palm) { + position: absolute; + top: 9px; + right: $spacing-unit / 2; + background-color: $background-color; + border: 1px solid $grey-color-light; + border-radius: 5px; + text-align: right; + + label[for="nav-trigger"] { + display: block; + float: right; + width: 36px; + height: 36px; + z-index: 2; + cursor: pointer; + } + + .menu-icon { + display: block; + float: right; + width: 36px; + height: 26px; + line-height: 0; + padding-top: 10px; + text-align: center; + + > svg path { + fill: $grey-color-dark; + } + } + + input ~ .trigger { + clear: both; + display: none; + } + + input:checked ~ .trigger { + display: block; + padding-bottom: 5px; + } + + .page-link { + display: block; + padding: 5px 10px; + + &:not(:last-child) { + margin-right: 0; + } + margin-left: 20px; + } + } +} + + + +/** + * Site footer + */ +.site-footer { + border-top: 1px solid $grey-color-light; + padding: $spacing-unit 0; + + color: $lightning-white; + background-color: $lightning-dkpurple; +} + +.footer-heading { + @include relative-font-size(1.125); + margin-bottom: $spacing-unit / 2; +} + +.contact-list, +.social-media-list { + list-style: none; + margin-left: 0; +} + +.footer-col-wrapper { + @include relative-font-size(0.9375); + // color: $grey-color; + margin-left: -$spacing-unit / 2; + @extend %clearfix; +} + +.footer-col { + float: left; + margin-bottom: $spacing-unit / 2; + padding-left: $spacing-unit / 2; +} + +.footer-col-1 { + width: -webkit-calc(35% - (#{$spacing-unit} / 2)); + width: calc(35% - (#{$spacing-unit} / 2)); +} + +.footer-col-2 { + width: -webkit-calc(20% - (#{$spacing-unit} / 2)); + width: calc(20% - (#{$spacing-unit} / 2)); +} + +.footer-col-3 { + width: -webkit-calc(45% - (#{$spacing-unit} / 2)); + width: calc(45% - (#{$spacing-unit} / 2)); +} + +@include media-query($on-laptop) { + .footer-col-1, + .footer-col-2 { + width: -webkit-calc(50% - (#{$spacing-unit} / 2)); + width: calc(50% - (#{$spacing-unit} / 2)); + } + + .footer-col-3 { + width: -webkit-calc(100% - (#{$spacing-unit} / 2)); + width: calc(100% - (#{$spacing-unit} / 2)); + } +} + +@include media-query($on-palm) { + .footer-col { + float: none; + width: -webkit-calc(100% - (#{$spacing-unit} / 2)); + width: calc(100% - (#{$spacing-unit} / 2)); + } +} + + + +/** + * Page content + */ +.page-content { + padding: $spacing-unit 0; +} + +.page-heading { + @include relative-font-size(2.75); + margin-top: 20px; + margin-bottom: 20px; +} + +.post-list { + margin-left: 0; + list-style: none; + + > li { + margin-bottom: $spacing-unit; + } +} + +.post-meta { + font-size: $small-font-size; + color: $grey-color; +} + +.post-link { + display: block; + @include relative-font-size(1.5); +} + + + +/** + * Posts + */ +.post-header { + margin-bottom: $spacing-unit; +} + +.post-title { + @include relative-font-size(2.625); + letter-spacing: -1px; + line-height: 1; + + @include media-query($on-laptop) { + @include relative-font-size(2.25); + } +} + +.post-content { + margin-bottom: $spacing-unit; + + h2 { + @include relative-font-size(2); + + @include media-query($on-laptop) { + @include relative-font-size(1.75); + } + } + + h3 { + @include relative-font-size(1.625); + + @include media-query($on-laptop) { + @include relative-font-size(1.375); + } + } + + h4 { + @include relative-font-size(1.25); + + @include media-query($on-laptop) { + @include relative-font-size(1.125); + } + } +} diff --git a/assets/dropdown.css b/assets/dropdown.css new file mode 100644 index 0000000..14fc0db --- /dev/null +++ b/assets/dropdown.css @@ -0,0 +1,49 @@ +/* Dropdown Button */ +.dropbtn { + background-color: #17062d; + color: white; + padding: 16px; + font-size: 16px; + border: none; + cursor: pointer; +} + +/* The container
- needed to position the dropdown content */ +.dropdown { + position: relative; + display: inline-block; +} + +/* Dropdown Content (Hidden by Default) */ +.dropdown-content { + display: none; + position: absolute; + background-color: #f9f9f9; + min-width: 130px; + box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2); + z-index: 1; +} + +/* Links inside the dropdown */ +.dropdown-content a { + color: black; + padding-left: 16px; + padding-right: 20px; + text-decoration: none; + display: block; +} + +/* Change color of dropdown links on hover */ +.dropdown-content a:hover { + background-color: #f1f1f1 +} + +/* Show the dropdown menu on hover */ +.dropdown:hover .dropdown-content { + display: block; +} + +/* Change the background color of the dropdown button when the dropdown content is shown */ +.dropdown:hover .dropbtn { + background-color: #17062d; +} diff --git a/assets/lapps/lnd.fun.png b/assets/lapps/lnd.fun.png new file mode 100644 index 0000000..395bb38 Binary files /dev/null and b/assets/lapps/lnd.fun.png differ diff --git a/assets/lapps/lnd_desktop_app.png b/assets/lapps/lnd_desktop_app.png new file mode 100644 index 0000000..69dbc51 Binary files /dev/null and b/assets/lapps/lnd_desktop_app.png differ diff --git a/assets/lapps/thunderdice.png b/assets/lapps/thunderdice.png new file mode 100644 index 0000000..abbb023 Binary files /dev/null and b/assets/lapps/thunderdice.png differ diff --git a/assets/lapps/yalls.png b/assets/lapps/yalls.png new file mode 100644 index 0000000..69f2666 Binary files /dev/null and b/assets/lapps/yalls.png differ diff --git a/assets/lapps/zap.png b/assets/lapps/zap.png new file mode 100644 index 0000000..c62e32a Binary files /dev/null and b/assets/lapps/zap.png differ diff --git a/deploy.sh b/deploy.sh new file mode 100755 index 0000000..c029919 --- /dev/null +++ b/deploy.sh @@ -0,0 +1,7 @@ +#!/bin/sh +jekyll build + +# -m use faster multithreaded uploads +# -d delete remote files that aren't in the source +# -r recurse into source subdirectories +gsutil -m rsync -d -r ./_site gs://dev.lightning.community diff --git a/_pages/docker-guide.md b/guides/docker.md similarity index 65% rename from _pages/docker-guide.md rename to guides/docker.md index 9613dc8..68dfa67 100644 --- a/_pages/docker-guide.md +++ b/guides/docker.md @@ -1,21 +1,20 @@ --- layout: page title: Working with LND and Docker -permalink: /docker-guide/ + --- -This document is written for people who are interested in working with the -Lightning Network Daemon (`lnd`) under a docker workflow. We use -`docker-compose` to package `lnd` and `btcd` together to make deploying the two -daemons as easy as typing a few commands. All configuration between `lnd` and -`btcd` are handled automatically by their `docker-compose` config file found -[here](https://github.com/lightningnetwork/lnd/tree/master/docker). +This document is written for people who are eager to do something with +the Lightning Network Daemon (`lnd`). This folder uses `docker-compose` to +package `lnd` and `btcd` together to make deploying the two daemons as easy as +typing a few commands. All configuration between `lnd` and `btcd` are handled +automatically by their `docker-compose` config file. ### Prerequisites - Name | Version - --------|--------- - docker-compose | 1.9.0 - docker | 1.13.0 +Name | Version +--------|--------- +docker-compose | 1.9.0 +docker | 1.13.0 ### Table of content * [Create lightning network cluster](#create-lightning-network-cluster) @@ -36,19 +35,19 @@ Current workflow is big because we recreate the whole network by ourselves, next versions will use the started `btcd` bitcoin node in `testnet` and `faucet` wallet from which you will get the bitcoins. -In the workflow below, we describe the steps required to recreate following +In the workflow below, we describe the steps required to recreate the following topology, and send a payment from `Alice` to `Bob`. ``` + ----- + + --- + | Alice | <--- channel ---> | Bob | <--- Bob and Alice are the lightning network daemons which -+ ----- + + --- + creates the channel and interact with each other using ++ ----- + + --- + create channels and interact with each other using the | | Bitcoin network as source of truth. | | + - - - - - + - - - - - - + | + --------------- + - | Bitcoin network | <--- In current scenario for simplicity we create only one - + --------------- + "btcd" node which represents the Bitcoin network, in + | Bitcoin network | <--- In the current scenario for simplicity we create only one + + --------------- + "btcd" node which represents the Bitcoin network, in a real situation Alice and Bob will likely be connected to different Bitcoin nodes. ``` @@ -68,25 +67,19 @@ Start `btcd`, and then create an address for `Alice` that we'll directly mine bitcoin into. ```bash # Init bitcoin network env variable: -$ export BITCOIN_NETWORK="simnet" - -# Run all commands from the Docker directory: -$ cd $GOPATH/src/github.com/lightningnetwork/lnd/docker - -# Run "btcd" node: -$ docker-compose up -d "btcd" +$ export NETWORK="simnet" # Run the "Alice" container and log into it: -$ docker-compose up -d "alice" -$ docker exec -i -t "alice" bash +$ docker-compose run -d --name alice lnd_btc +$ docker exec -i -t alice bash # Generate a new backward compatible nested p2sh address for Alice: -alice$ lncli newaddress np2wkh +alice$ lncli --network=simnet newaddress np2wkh # Recreate "btcd" node and set Alice's address as mining address: -$ MINING_ADDRESS= docker-compose up -d "btcd" +$ MINING_ADDRESS= docker-compose up -d btcd -# Generate 400 block (we need at least "100 >=" blocks because of coinbase +# Generate 400 blocks (we need at least "100 >=" blocks because of coinbase # block maturity and "300 ~=" in order to activate segwit): $ docker-compose run btcctl generate 400 @@ -96,24 +89,25 @@ $ docker-compose run btcctl getblockchaininfo | grep -A 1 segwit Check `Alice` balance: ``` -alice$ lncli walletbalance --witness_only=true +alice$ lncli --network=simnet walletbalance ``` Connect `Bob` node to `Alice` node. ```bash # Run "Bob" node and log into it: -$ docker-compose up --no-recreate -d "bob" -$ docker exec -i -t "bob" bash +$ docker-compose run -d --name bob lnd_btc +$ docker exec -i -t bob bash # Get the identity pubkey of "Bob" node: -bob$ lncli getinfo +bob$ lncli --network=simnet getinfo { ----->"identity_pubkey": "0343bc80b914aebf8e50eb0b8e445fc79b9e6e8e5e018fa8c5f85c7d429c117b38", "alias": "", "num_pending_channels": 0, "num_active_channels": 0, + "num_inactive_channels": 0, "num_peers": 0, "block_height": 1215, "block_hash": "7d0bc86ea4151ed3b5be908ea883d2ac3073263537bcf8ca2dca4bec22e79d50", @@ -125,19 +119,18 @@ bob$ lncli getinfo } # Get the IP address of "Bob" node: -$ docker inspect "bob" | grep IPAddress +$ docker inspect bob | grep IPAddress # Connect "Alice" to the "Bob" node: -alice$ lncli connect @ +alice$ lncli --network=simnet connect @ # Check list of peers on "Alice" side: -alice$ lncli listpeers +alice$ lncli --network=simnet listpeers { "peers": [ { "pub_key": "0343bc80b914aebf8e50eb0b8e445fc79b9e6e8e5e018fa8c5f85c7d429c117b38", - "peer_id": 1, - "address": "172.19.0.4:5656", + "address": "172.19.0.4:9735", "bytes_sent": "357", "bytes_recv": "357", "sat_sent": "0", @@ -149,12 +142,11 @@ alice$ lncli listpeers } # Check list of peers on "Bob" side: -bob$ lncli listpeers +bob$ lncli --network=simnet listpeers { "peers": [ { "pub_key": "03d0cd35b761f789983f3cfe82c68170cd1c3266b39220c24f7dd72ef4be0883eb", - "peer_id": 1, "address": "172.19.0.3:51932", "bytes_sent": "357", "bytes_recv": "357", @@ -170,13 +162,13 @@ bob$ lncli listpeers Create the `Alice<->Bob` channel. ```bash # Open the channel with "Bob": -alice$ lncli openchannel --node_key= --num_confs=1 --local_amt=1000000 +alice$ lncli --network=simnet openchannel --node_key= --local_amt=1000000 -# Include funding transaction in block thereby open the channel: -$ docker-compose run btcctl generate 1 +# Include funding transaction in block thereby opening the channel: +$ docker-compose run btcctl generate 3 -# Check that channel with "Bob" was created: -alice$ lncli listchannels +# Check that channel with "Bob" was opened: +alice$ lncli --network=simnet listchannels { "channels": [ { @@ -187,42 +179,47 @@ alice$ lncli listchannels "capacity": "1005000", "local_balance": "1000000", "remote_balance": "0", + "commit_fee": "8688", + "commit_weight": "600", + "fee_per_kw": "12000", "unsettled_balance": "0", "total_satoshis_sent": "0", "total_satoshis_received": "0", - "num_updates": "0" + "num_updates": "0", + "pending_htlcs": [ + ], + "csv_delay": 4 } ] } ``` -Send the payment form `Alice` to `Bob`. +Send the payment from `Alice` to `Bob`. ```bash # Add invoice on "Bob" side: -bob$ lncli addinvoice --value=10000 +bob$ lncli --network=simnet addinvoice --amt=10000 { "r_hash": "", "pay_req": "", } # Send payment from "Alice" to "Bob": -alice$ lncli sendpayment --pay_req= +alice$ lncli --network=simnet sendpayment --pay_req= -# Check "Alice"'s channel balance was decremented accordingly by the payment -# amount -alice$ lncli listchannels +# Check "Alice"'s channel balance +alice$ lncli --network=simnet channelbalance -# Check "Bob"'s channel balance was credited with the payment amount -bob$ lncli listchannels +# Check "Bob"'s channel balance +bob$ lncli --network=simnet channelbalance ``` Now we have open channel in which we sent only one payment, let's imagine -that we sent lots of them and we'll now like to close the channel. Let's do +that we sent lots of them and we'd now like to close the channel. Let's do it! ```bash -# List the "Alice" channel and retrieve "channel_point" which represent +# List the "Alice" channel and retrieve "channel_point" which represents # the opened channel: -alice$ lncli listchannels +alice$ lncli --network=simnet listchannels { "channels": [ { @@ -233,40 +230,47 @@ alice$ lncli listchannels "capacity": "1005000", "local_balance": "990000", "remote_balance": "10000", + "commit_fee": "8688", + "commit_weight": "724", + "fee_per_kw": "12000", "unsettled_balance": "0", "total_satoshis_sent": "10000", "total_satoshis_received": "0", - "num_updates": "2" + "num_updates": "2", + "pending_htlcs": [ + ], + "csv_delay": 4 } ] } -# Channel point consist of two numbers separated by colon the first one +# Channel point consists of two numbers separated by a colon. The first one # is "funding_txid" and the second one is "output_index": -alice$ lncli closechannel --funding_txid= --output_index= +alice$ lncli --network=simnet closechannel --funding_txid= --output_index= -# Include close transaction in block thereby close the channel: -$ docker-compose run btcctl generate 1 +# Include close transaction in a block thereby closing the channel: +$ docker-compose run btcctl generate 3 # Check "Alice" on-chain balance was credited by her settled amount in the channel: -alice$ lncli walletbalance +alice$ lncli --network=simnet walletbalance # Check "Bob" on-chain balance was credited with the funds he received in the # channel: -bob$ lncli walletbalance +bob$ lncli --network=simnet walletbalance { - "balance": 0.0001 + "total_balance": "10000", + "confirmed_balance": "10000", + "unconfirmed_balance": "0" } ``` ### Connect to faucet lightning node -In order to be more confident with `lnd` commands perhaps try to create a mini -lightning network cluster ([Create lightning network -cluster](#create-lightning-network-cluster)). +In order to be more confident with `lnd` commands I suggest you to try +to create a mini lightning network cluster ([Create lightning network cluster](#create-lightning-network-cluster)). -In this section we will try to connect our node to the faucet/hub node which -will create with as the channel and send as some amount of bitcoins. The schema -will be following: +In this section we will try to connect our node to the faucet/hub node +which we will create a channel with and send some amount of +bitcoins. The schema will be following: ``` + ----- + + ------ + (1) + --- + @@ -281,38 +285,41 @@ will be following: + --------------- + - (1) You may connect an additinal node "Bob" and make the multihope + (1) You may connect an additional node "Bob" and make the multihop payment Alice->Faucet->Bob (2) "Faucet", "Alice" and "Bob" are the lightning network daemons which - creates the channel and interact with each other using Bitcoin network + create channels to interact with each other using the Bitcoin network as source of truth. (3) In current scenario "Alice" and "Faucet" lightning network nodes - connected to different Bitcoin nodes. If you decide to connect "Bob" - to "Faucet" than already created "btcd" node would be sufficient. + connect to different Bitcoin nodes. If you decide to connect "Bob" + to "Faucet" then the already created "btcd" node would be sufficient. ``` -First of all you need to run `btcd` node in `testnet` and wait it to be -synced with test network (`May the Force and Patience be with you` ᕦ(ò_óˇ)ᕤ). +First of all you need to run `btcd` node in `testnet` and wait for it to be +synced with test network (`May the Force and Patience be with you`). ```bash # Init bitcoin network env variable: -$ export BITCOIN_NETWORK="testnet" +$ export NETWORK="testnet" # Run "btcd" node: $ docker-compose up -d "btcd" ``` After `btcd` synced, connect `Alice` to the `Faucet` node. + +The `Faucet` node address can be found at the [Faucet Lightning Community webpage](https://faucet.lightning.community). + ```bash # Run "Alice" container and log into it: -$ docker-compose up -d "alice"; docker exec -i -t "alice" bash +$ docker-compose run -d --name alice lnd_btc; docker exec -i -t "alice" bash # Connect "Alice" to the "Faucet" node: -alice$ lncli connect @ +alice$ lncli --network=testnet connect @ ``` -After connection was achieved the `Faucet` node should create the channel +After a connection is achieved, the `Faucet` node should create the channel and send some amount of bitcoins to `Alice`. **What you may do next?:** @@ -321,10 +328,11 @@ and send some amount of bitcoins to `Alice`. - Close channel with `Faucet` and check the onchain balance. ### Questions -[![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)] -(https://webchat.freenode.net/?channels=lnd) +[![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)](https://webchat.freenode.net/?channels=lnd) * How to see `alice` | `bob` | `btcd` logs? ```bash docker-compose logs ``` + + diff --git a/guides/index.md b/guides/index.md new file mode 100644 index 0000000..d3c1399 --- /dev/null +++ b/guides/index.md @@ -0,0 +1,38 @@ +--- +layout: page +title: Guides +--- + +This page lists the writeups available on specific skills in and around LND. + +#### LND Overview and Developer Guide + +Modeled after the Bitcoin Developer Guide, the [LND Overview and Developer +Guide](/overview/) will provide the necessary conceptual background for +developers looking to build on LND. + +#### Installation + +The [installation guide](/guides/installation/) will get you set up to build and +run `lnd` from the command line. + +#### Tutorial + +The [tutorial](/tutorial/) is a holistic tour that is intended to teach all the +dev-specific skills necessary to work with `lnd`, by walking through a setup of +`lnd`, a lnd client, RPC, and integration into a web application. This should be +completed in tandem with the [LND Overview and Developer Guide](/overview/). + +#### gRPC Guides + +The gRPC guides will get you started sending command through gRPC. We have +written language specific guides for Python and Javascript: + +* [Python](/guides/python-grpc/) +* [Javascript](/guides/javascript-grpc/) + +#### Docker + +The [Docker guide](/guides/docker/) teaches you how to package `lnd` and `btcd` +together to make deployment of multiple daemons a bit simpler. + diff --git a/guides/installation.md b/guides/installation.md new file mode 100644 index 0000000..f8feb31 --- /dev/null +++ b/guides/installation.md @@ -0,0 +1,475 @@ +--- +layout: page +title: Installation + +--- + +* [Installation](#installation) + * [Preliminaries](#preliminaries) + * [Installing lnd](#installing-lnd) +* [Available Backend Operating Modes](#available-backend-operating-modes) + * [btcd Options](#btcd-options) + * [Neutrino Options](#neutrino-options) + * [Bitcoind Options](#bitcoind-options) + * [Using btcd](#using-btcd) + * [Installing btcd](#installing-btcd) + * [Starting btcd](#starting-btcd) + * [Running lnd using the btcd backend](#running-lnd-using-the-btcd-backend) + * [Using Neutrino](#using-neutrino) + * [Using bitcoind or litecoind](#using-bitcoind-or-litecoind) +* [Creating a Wallet](#creating-a-wallet) +* [Macaroons](#macaroons) +* [Network Reachability](#network-reachability) +* [Simnet vs. Testnet Development](#simnet-vs-testnet-development) +* [Creating an lnd.conf (Optional)](#creating-an-lndconf-optional) + +# Installation + +### Preliminaries + In order to work with [`lnd`](https://github.com/lightningnetwork/lnd), the + following build dependencies are required: + + * **Go:** `lnd` is written in Go. To install, run one of the following commands: + + + **Note**: The minimum version of Go supported is Go 1.13. We recommend that + users use the latest version of Go, which at the time of writing is + [`1.13`](https://blog.golang.org/go1.13). + + + On Linux: + + (x86-64) + ``` + wget https://dl.google.com/go/go1.13.linux-amd64.tar.gz + sha256sum go1.13.linux-amd64.tar.gz | awk -F " " '{ print $1 }' + ``` + + The final output of the command above should be + `68a2297eb099d1a76097905a2ce334e3155004ec08cdea85f24527be3c48e856`. If it + isn't, then the target REPO HAS BEEN MODIFIED, and you shouldn't install + this version of Go. If it matches, then proceed to install Go: + ``` + tar -C /usr/local -xzf go1.13.linux-amd64.tar.gz + export PATH=$PATH:/usr/local/go/bin + ``` + + (ARMv6) + ``` + wget https://dl.google.com/go/go1.13.linux-armv6l.tar.gz + sha256sum go1.13.linux-armv6l.tar.gz | awk -F " " '{ print $1 }' + ``` + + The final output of the command above should be + `931906d67cae1222f501e7be26e0ee73ba89420be0c4591925901cb9a4e156f0`. If it + isn't, then the target REPO HAS BEEN MODIFIED, and you shouldn't install + this version of Go. If it matches, then proceed to install Go: + ``` + tar -C /usr/local -xzf go1.13.linux-armv6l.tar.gz + export PATH=$PATH:/usr/local/go/bin + ``` + + On Mac OS X: + ``` + brew install go@1.13 + ``` + + On FreeBSD: + ``` + pkg install go + ``` + + Alternatively, one can download the pre-compiled binaries hosted on the + [Golang download page](https://golang.org/dl/). If one seeks to install + from source, then more detailed installation instructions can be found + [here](https://golang.org/doc/install). + + At this point, you should set your `$GOPATH` environment variable, which + represents the path to your workspace. By default, `$GOPATH` is set to + `~/go`. You will also need to add `$GOPATH/bin` to your `PATH`. This ensures + that your shell will be able to detect the binaries you install. + + ```bash + export GOPATH=~/gocode + export PATH=$PATH:$GOPATH/bin + ``` + + We recommend placing the above in your .bashrc or in a setup script so that + you can avoid typing this every time you open a new terminal window. + + * **Go modules:** This project uses [Go modules](https://github.com/golang/go/wiki/Modules) + to manage dependencies as well as to provide *reproducible builds*. + + Usage of Go modules (with Go 1.12) means that you no longer need to clone + `lnd` into your `$GOPATH` for development purposes. Instead, your `lnd` + repo can now live anywhere! + +### Installing lnd + +With the preliminary steps completed, to install `lnd`, `lncli`, and all +related dependencies run the following commands: +``` +go get -d github.com/lightningnetwork/lnd +cd $GOPATH/src/github.com/lightningnetwork/lnd +make && make install +``` + +**NOTE**: Our instructions still use the `$GOPATH` directory from prior +versions of Go, but with Go 1.12, it's now possible for `lnd` to live +_anywhere_ on your file system. + +For Windows WSL users, make will need to be referenced directly via +/usr/bin/make/, or alternatively by wrapping quotation marks around make, +like so: + +``` +/usr/bin/make && /usr/bin/make install + +"make" && "make" install +``` + +On FreeBSD, use gmake instead of make. + +Alternatively, if one doesn't wish to use `make`, then the `go` commands can be +used directly: +``` +GO111MODULE=on go install -v ./... +``` + +**Updating** + +To update your version of `lnd` to the latest version run the following +commands: +``` +cd $GOPATH/src/github.com/lightningnetwork/lnd +git pull +make clean && make && make install +``` + +On FreeBSD, use gmake instead of make. + +Alternatively, if one doesn't wish to use `make`, then the `go` commands can be +used directly: +``` +cd $GOPATH/src/github.com/lightningnetwork/lnd +git pull +GO111MODULE=on go install -v ./... +``` + +**Tests** + +To check that `lnd` was installed properly run the following command: +``` +make check +``` + +# Available Backend Operating Modes + +In order to run, `lnd` requires, that the user specify a chain backend. At the +time of writing of this document, there are three available chain backends: +`btcd`, `neutrino`, `bitcoind`. All but neutrino (atm) can run on mainnet with +an out of the box `lnd` instance. We don't require `--txindex` when running +with `bitcoind` or `btcd` but activating the `txindex` will generally make +`lnd` run faster. + +**NOTE: WE DO NOT FULLY SUPPORT PRUNED OPERATING MODES FOR FULL NODES.** It's +possible to run a node in a pruned mode and have it serve lnd, however one must +take care to ensure that `lnd` has all blocks on disk since the birth of the +wallet, and the age of the earliest channels (which were created around March +2018). + +The set of arguments for each of the backend modes is as follows: + +## btcd Options +``` +btcd: + --btcd.dir= The base directory that contains the node's data, logs, configuration file, etc. (default: /Users/roasbeef/Library/Application Support/Btcd) + --btcd.rpchost= The daemon's rpc listening address. If a port is omitted, then the default port for the selected chain parameters will be used. (default: localhost) + --btcd.rpcuser= Username for RPC connections + --btcd.rpcpass= Password for RPC connections + --btcd.rpccert= File containing the daemon's certificate file (default: /Users/roasbeef/Library/Application Support/Btcd/rpc.cert) + --btcd.rawrpccert= The raw bytes of the daemon's PEM-encoded certificate chain which will be used to authenticate the RPC connection. +``` + +## Neutrino Options +``` +neutrino: + -a, --neutrino.addpeer= Add a peer to connect with at startup + --neutrino.connect= Connect only to the specified peers at startup + --neutrino.maxpeers= Max number of inbound and outbound peers + --neutrino.banduration= How long to ban misbehaving peers. Valid time units are {s, m, h}. Minimum 1 second + --neutrino.banthreshold= Maximum allowed ban score before disconnecting and banning misbehaving peers. +``` + +## Bitcoind Options +``` +bitcoind: + --bitcoind.dir= The base directory that contains the node's data, logs, configuration file, etc. (default: /Users/roasbeef/Library/Application Support/Bitcoin) + --bitcoind.rpchost= The daemon's rpc listening address. If a port is omitted, then the default port for the selected chain parameters will be used. (default: localhost) + --bitcoind.rpcuser= Username for RPC connections + --bitcoind.rpcpass= Password for RPC connections + --bitcoind.zmqpubrawblock= The address listening for ZMQ connections to deliver raw block notifications + --bitcoind.zmqpubrawtx= The address listening for ZMQ connections to deliver raw transaction notifications +``` + +## Using btcd + +### Installing btcd + +On FreeBSD, use gmake instead of make. + +To install btcd, run the following commands: + +Install **btcd**: +``` +make btcd +``` + +Alternatively, you can install [`btcd` directly from its +repo](https://github.com/btcsuite/btcd). + +### Starting btcd + +Running the following command will create `rpc.cert` and default `btcd.conf`. + +``` +btcd --testnet --rpcuser=REPLACEME --rpcpass=REPLACEME +``` +If you want to use `lnd` on testnet, `btcd` needs to first fully sync the +testnet blockchain. Depending on your hardware, this may take up to a few +hours. Note that adding `--txindex` is optional, as it will take longer to sync +the node, but then `lnd` will generally operate faster as it can hit the index +directly, rather than scanning blocks or BIP 158 filters for relevant items. + +(NOTE: It may take several minutes to find segwit-enabled peers.) + +While `btcd` is syncing you can check on its progress using btcd's `getinfo` +RPC command: +``` +btcctl --testnet --rpcuser=REPLACEME --rpcpass=REPLACEME getinfo +{ + "version": 120000, + "protocolversion": 70002, + "blocks": 1114996, + "timeoffset": 0, + "connections": 7, + "proxy": "", + "difficulty": 422570.58270815, + "testnet": true, + "relayfee": 0.00001, + "errors": "" +} +``` + +Additionally, you can monitor btcd's logs to track its syncing progress in real +time. + +You can test your `btcd` node's connectivity using the `getpeerinfo` command: +``` +btcctl --testnet --rpcuser=REPLACEME --rpcpass=REPLACEME getpeerinfo | more +``` + +### Running lnd using the btcd backend + +If you are on testnet, run this command after `btcd` has finished syncing. +Otherwise, replace `--bitcoin.testnet` with `--bitcoin.simnet`. If you are +installing `lnd` in preparation for the +[tutorial](https://dev.lightning.community/tutorial), you may skip this step. +``` +lnd --bitcoin.active --bitcoin.testnet --debuglevel=debug --btcd.rpcuser=kek --btcd.rpcpass=kek --externalip=X.X.X.X +``` + +## Using Neutrino + +In order to run `lnd` in its light client mode, you'll need to locate a +full-node which is capable of serving this new light client mode. `lnd` uses +[BIP 157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki) and [BIP +158](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki) for its light client +mode. A public instance of such a node can be found at +`faucet.lightning.community`. + +To run lnd in neutrino mode, run `lnd` with the following arguments, (swapping +in `--bitcoin.simnet` if needed), and also your own `btcd` node if available: +``` +lnd --bitcoin.active --bitcoin.testnet --debuglevel=debug --bitcoin.node=neutrino --neutrino.connect=faucet.lightning.community +``` + + +## Using bitcoind or litecoind + +The configuration for bitcoind and litecoind are nearly identical, the +following steps can be mirrored with loss of generality to enable a litecoind +backend. Setup will be described in regards to `bitcoind`, but note that `lnd` +uses a distinct `litecoin.node=litecoind` argument and analogous +subconfigurations prefixed by `litecoind`. Note that adding `--txindex` is +optional, as it will take longer to sync the node, but then `lnd` will +generally operate faster as it can hit the index directly, rather than scanning +blocks or BIP 158 filters for relevant items. + +To configure your bitcoind backend for use with lnd, first complete and verify +the following: + +- Since `lnd` uses + [ZeroMQ](https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md) to + interface with `bitcoind`, *your `bitcoind` installation must be compiled with + ZMQ*. Note that if you installed `bitcoind` from source and ZMQ was not present, + then ZMQ support will be disabled, and `lnd` will quit on a `connection refused` error. + If you installed `bitcoind` via Homebrew in the past ZMQ may not be included + ([this has now been fixed](https://github.com/Homebrew/homebrew-core/pull/23088) + in the latest Homebrew recipe for bitcoin) +- Configure the `bitcoind` instance for ZMQ with `--zmqpubrawblock` and + `--zmqpubrawtx`. These options must each use their own unique address in order + to provide a reliable delivery of notifications (e.g. + `--zmqpubrawblock=tcp://127.0.0.1:28332` and + `--zmqpubrawtx=tcp://127.0.0.1:28333`). +- Start `bitcoind` running against testnet, and let it complete a full sync with + the testnet chain (alternatively, use `--bitcoind.regtest` instead). + +Here's a sample `bitcoin.conf` for use with lnd: +``` +testnet=1 +server=1 +daemon=1 +zmqpubrawblock=tcp://127.0.0.1:28332 +zmqpubrawtx=tcp://127.0.0.1:28333 +``` + +Once all of the above is complete, and you've confirmed `bitcoind` is fully +updated with the latest blocks on testnet, run the command below to launch +`lnd` with `bitcoind` as your backend (as with `bitcoind`, you can create an +`lnd.conf` to save these options, more info on that is described further +below): + +``` +lnd --bitcoin.active --bitcoin.testnet --debuglevel=debug --bitcoin.node=bitcoind --bitcoind.rpcuser=REPLACEME --bitcoind.rpcpass=REPLACEME --bitcoind.zmqpubrawblock=tcp://127.0.0.1:28332 --bitcoind.zmqpubrawtx=tcp://127.0.0.1:28333 --externalip=X.X.X.X +``` + +*NOTE:* +- The auth parameters `rpcuser` and `rpcpass` parameters can typically be + determined by `lnd` for a `bitcoind` instance running under the same user, + including when using cookie auth. In this case, you can exclude them from the + `lnd` options entirely. +- If you DO choose to explicitly pass the auth parameters in your `lnd.conf` or + command line options for `lnd` (`bitcoind.rpcuser` and `bitcoind.rpcpass` as + shown in example command above), you must also specify the + `bitcoind.zmqpubrawblock` and `bitcoind.zmqpubrawtx` options. Otherwise, `lnd` + will attempt to get the configuration from your `bitcoin.conf`. +- You must ensure the same addresses are used for the `bitcoind.zmqpubrawblock` + and `bitcoind.zmqpubrawtx` options passed to `lnd` as for the `zmqpubrawblock` + and `zmqpubrawtx` passed in the `bitcoind` options respectively. +- When running lnd and bitcoind on the same Windows machine, ensure you use + 127.0.0.1, not localhost, for all configuration options that require a TCP/IP + host address. If you use "localhost" as the host name, you may see extremely + slow inter-process-communication between lnd and the bitcoind backend. If lnd + is experiencing this issue, you'll see "Waiting for chain backend to finish + sync, start_height=XXXXXX" as the last entry in the console or log output, and + lnd will appear to hang. Normal lnd output will quickly show multiple + messages like this as lnd consumes blocks from bitcoind. +- Don't connect more than two or three instances of `lnd` to `bitcoind`. With + the default `bitcoind` settings, having more than one instance of `lnd`, or + `lnd` plus any application that consumes the RPC could cause `lnd` to miss + crucial updates from the backend. + + +# Creating a wallet +If `lnd` is being run for the first time, create a new wallet with: +``` +lncli create +``` +This will prompt for a wallet password, and optionally a cipher seed +passphrase. + +`lnd` will then print a 24 word cipher seed mnemonic, which can be used to +recover the wallet in case of data loss. The user should write this down and +keep in a safe place. + + +# Macaroons + +`lnd`'s authentication system is called **macaroons**, which are decentralized +bearer credentials allowing for delegation, attenuation, and other cool +features. You can learn more about them in Alex Akselrod's [writeup on +Github](https://github.com/lightningnetwork/lnd/issues/20). + +Running `lnd` for the first time will by default generate the `admin.macaroon`, +`read_only.macaroon`, and `macaroons.db` files that are used to authenticate +into `lnd`. They will be stored in the network directory (default: +`lnddir/data/chain/bitcoin/mainnet`) so that it's possible to use a distinct +password for mainnet, testnet, simnet, etc. Note that if you specified an +alternative data directory (via the `--datadir` argument), you will have to +additionally pass the updated location of the `admin.macaroon` file into `lncli` +using the `--macaroonpath` argument. + +To disable macaroons for testing, pass the `--no-macaroons` flag into *both* +`lnd` and `lncli`. + +# Network Reachability + +If you'd like to signal to other nodes on the network that you'll accept +incoming channels (as peers need to connect inbound to initiate a channel +funding workflow), then the `--externalip` flag should be set to your publicly +reachable IP address. + +# Simnet vs. Testnet Development + +If you are doing local development, such as for the tutorial, you'll want to +start both `btcd` and `lnd` in the `simnet` mode. Simnet is similar to regtest +in that you'll be able to instantly mine blocks as needed to test `lnd` +locally. In order to start either daemon in the `simnet` mode use `simnet` +instead of `testnet`, adding the `--bitcoin.simnet` flag instead of the +`--bitcoin.testnet` flag. + +Another relevant command line flag for local testing of new `lnd` developments +is the `--debughtlc` flag. When starting `lnd` with this flag, it'll be able to +automatically settle a special type of HTLC sent to it. This means that you +won't need to manually insert invoices in order to test payment connectivity. +To send this "special" HTLC type, include the `--debugsend` command at the end +of your `sendpayment` commands. + + +There are currently two primary ways to run `lnd`: one requires a local `btcd` +instance with the RPC service exposed, and the other uses a fully integrated +light client powered by [neutrino](https://github.com/lightninglabs/neutrino). + +# Creating an lnd.conf (Optional) + +Optionally, if you'd like to have a persistent configuration between `lnd` +launches, allowing you to simply type `lnd --bitcoin.testnet --bitcoin.active` +at the command line, you can create an `lnd.conf`. + +**On MacOS, located at:** +`/Users/[username]/Library/Application Support/Lnd/lnd.conf` + +**On Linux, located at:** +`~/.lnd/lnd.conf` + +Here's a sample `lnd.conf` for `btcd` to get you started: +``` +[Application Options] +debuglevel=trace +maxpendingchannels=10 + +[Bitcoin] +bitcoin.active=1 +``` + +Notice the `[Bitcoin]` section. This section houses the parameters for the +Bitcoin chain. `lnd` also supports Litecoin testnet4 (but not both BTC and LTC +at the same time), so when working with Litecoin be sure to set to parameters +for Litecoin accordingly. See a more detailed sample config file available +[here](https://github.com/lightningnetwork/lnd/blob/master/sample-lnd.conf) +and explore the other sections for node configuration, including `[Btcd]`, +`[Bitcoind]`, `[Neutrino]`, `[Ltcd]`, and `[Litecoind]` depending on which +chain and node type you're using. + + + + +### Next Steps + +* **[Tutorial](/tutorial/):** Get acquainted with the skills necessary for `lnd` development. +* **[Developer Guides](/guides/):** Look through developer manuals on gRPC, + Docker, and more. +* **[Resources](/resources/):** Learn about the Lightning Network +* **[Code Contribution Guidelines](/contribute/):** Contribute to `lnd` itself. + diff --git a/guides/javascript-grpc.md b/guides/javascript-grpc.md new file mode 100644 index 0000000..59dd88a --- /dev/null +++ b/guides/javascript-grpc.md @@ -0,0 +1,242 @@ +--- +layout: page +title: How to write a Javascript gRPC client for the Lightning Network Daemon + +--- + +### Setup and Installation + +First, you'll need to initialize a simple nodejs project: +``` +npm init (or npm init -f if you want to use the default values without prompt) +``` + +Then you need to install the Javascript grpc library dependency: +``` +npm install grpc --save +``` + +You also need to copy the `lnd` `rpc.proto` file in your project directory (or +at least somewhere reachable by your Javascript code). + +The `rpc.proto` file is [located in the `lnrpc` directory of the `lnd` +sources](https://github.com/lightningnetwork/lnd/blob/master/lnrpc/rpc.proto). + +In order for the auto-generated code to compile successfully, you'll need to +comment out the following line: + +``` +//import "google/api/annotations.proto"; +``` + +#### Imports and Client + +Every time you work with Javascript gRPC, you will have to import `grpc`, load +`rpc.proto`, and create a connection to your client like so: + +```js +var grpc = require('grpc'); +var fs = require("fs"); + +// Due to updated ECDSA generated tls.cert we need to let gprc know that +// we need to use that cipher suite otherwise there will be a handhsake +// error when we communicate with the lnd rpc server. +process.env.GRPC_SSL_CIPHER_SUITES = 'HIGH+ECDSA' + +// Lnd cert is at ~/.lnd/tls.cert on Linux and +// ~/Library/Application Support/Lnd/tls.cert on Mac +var lndCert = fs.readFileSync("~/.lnd/tls.cert"); +var credentials = grpc.credentials.createSsl(lndCert); +var lnrpcDescriptor = grpc.load("rpc.proto"); +var lnrpc = lnrpcDescriptor.lnrpc; +var lightning = new lnrpc.Lightning('localhost:10009', credentials); +``` + +### Examples + +Let's walk through some examples of Javascript gRPC clients. These examples +assume that you have at least two `lnd` nodes running, the RPC location of one +of which is at the default `localhost:10009`, with an open channel between the +two nodes. + +#### Simple RPC + +```js +> lightning.getInfo({}, function(err, response) { + console.log('GetInfo:', response); + }); +``` + +You should get something like this in your console: + +``` +GetInfo: { identity_pubkey: '03c892e3f3f077ea1e381c081abb36491a2502bc43ed37ffb82e264224f325ff27', + alias: '', + num_pending_channels: 0, + num_active_channels: 1, + num_inactive_channels: 0, + num_peers: 1, + block_height: 1006, + block_hash: '198ba1dc43b4190e507fa5c7aea07a74ec0009a9ab308e1736dbdab5c767ff8e', + synced_to_chain: false, + testnet: false, + chains: [ 'bitcoin' ] } +``` + +#### Response-streaming RPC + +```js +var call = lightning.subscribeInvoices({}); +call.on('data', function(invoice) { + console.log(invoice); +}) +.on('end', function() { + // The server has finished sending +}) +.on('status', function(status) { + // Process status + console.log("Current status" + status); +}); +``` + +Now, create an invoice for your node at `localhost:10009`and send a payment to +it from another node. +```bash +$ lncli addinvoice --amt=100 +{ + "r_hash": , + "pay_req": +} +$ lncli sendpayment --pay_req= +``` +Your Javascript console should now display the details of the recently satisfied +invoice. + +#### Bidirectional-streaming RPC + +This example has a few dependencies: +```shell +npm install --save async lodash bytebuffer +``` + +You can run the following in your shell or put it in a program and run it like +`node script.js` + +```js +// Load some libraries specific to this example +var async = require('async'); +var _ = require('lodash'); +var ByteBuffer = require('bytebuffer'); + +var dest_pubkey = ; +var dest_pubkey_bytes = ByteBuffer.fromHex(dest_pubkey); + +// Set a listener on the bidirectional stream +var call = lightning.sendPayment(); +call.on('data', function(payment) { + console.log("Payment sent:"); + console.log(payment); +}); +call.on('end', function() { + // The server has finished + console.log("END"); +}); + +// You can send single payments like this +call.write({ dest: dest_pubkey_bytes, amt: 6969 }); + +// Or send a bunch of them like this +function paymentSender(destination, amount) { + return function(callback) { + console.log("Sending " + amount + " satoshis"); + console.log("To: " + destination); + call.write({ + dest: destination, + amt: amount + }); + _.delay(callback, 2000); + }; +} +var payment_senders = []; +for (var i = 0; i < 10; i++) { + payment_senders[i] = paymentSender(dest_pubkey_bytes, 100); +} +async.series(payment_senders, function() { + call.end(); +}); + +``` +This example will send a payment of 100 satoshis every 2 seconds. + + +#### Using Macaroons + +To authenticate using macaroons you need to include the macaroon in the metadata of the request. + +```js +var fs = require('fs'); +var grpc = require('grpc'); + +process.env.GRPC_SSL_CIPHER_SUITES = 'HIGH+ECDSA' + +// Lnd admin macaroon is at ~/.lnd/data/chain/bitcoin/simnet/admin.macaroon on Linux and +// ~/Library/Application Support/Lnd/data/chain/bitcoin/simnet/admin.macaroon on Mac +var m = fs.readFileSync('~/.lnd/data/chain/bitcoin/simnet/admin.macaroon'); +var macaroon = m.toString('hex'); +var meta = new grpc.Metadata().add('macaroon', macaroon); + +var lnrpcDescriptor = grpc.load("rpc.proto"); +var lnrpc = lnrpcDescriptor.lnrpc; +var client = new lnrpc.Lightning('some.address:10009', grpc.credentials.createInsecure()); + +client.getInfo({}, meta); +``` + +However, this can get tiresome to do for each request, so to avoid explicitly including the macaroon we can update the credentials to include it automatically. + +```js +var fs = require('fs'); +var grpc = require('grpc'); + +process.env.GRPC_SSL_CIPHER_SUITES = 'HIGH+ECDSA' + +// Lnd admin macaroon is at ~/.lnd/data/chain/bitcoin/simnet/admin.macaroon on Linux and +// ~/Library/Application Support/Lnd/data/chain/bitcoin/simnet/admin.macaroon on Mac +var m = fs.readFileSync('~/.lnd/data/chain/bitcoin/simnet/admin.macaroon'); +var macaroon = m.toString('hex'); + +// build meta data credentials +var metadata = new grpc.Metadata() +metadata.add('macaroon', macaroon) +var macaroonCreds = grpc.credentials.createFromMetadataGenerator((_args, callback) => { + callback(null, metadata); +}); + +// build ssl credentials using the cert the same as before +var lndCert = fs.readFileSync("~/.lnd/tls.cert"); +var sslCreds = grpc.credentials.createSsl(lndCert); + +// combine the cert credentials and the macaroon auth credentials +// such that every call is properly encrypted and authenticated +var credentials = grpc.credentials.combineChannelCredentials(sslCreds, macaroonCreds); + +// Pass the crendentials when creating a channel +var lnrpcDescriptor = grpc.load("rpc.proto"); +var lnrpc = lnrpcDescriptor.lnrpc; +var client = new lnrpc.Lightning('some.address:10009', credentials); + +client.getInfo({}, (err, res) => { ... }); +``` + + +### Conclusion + +With the above, you should have all the `lnd` related `gRPC` dependencies +installed locally in your project. In order to get up to speed with `protobuf` +usage from Javascript, see [this official `protobuf` reference for +Javascript](https://developers.google.com/protocol-buffers/docs/reference/javascript-generated). +Additionally, [this official gRPC +resource](http://www.grpc.io/docs/tutorials/basic/node.html) provides more +details around how to drive `gRPC` from `node.js`. + + diff --git a/guides/python-grpc.md b/guides/python-grpc.md new file mode 100644 index 0000000..9c8b7d4 --- /dev/null +++ b/guides/python-grpc.md @@ -0,0 +1,197 @@ +--- +layout: page +title: How to write a Python gRPC client for the Lightning Network Daemon + +--- + +This section enumerates what you need to do to write a client that communicates +with `lnd` in Python. + +### Setup and Installation + +Lnd uses the gRPC protocol for communication with clients like lncli. gRPC is +based on protocol buffers and as such, you will need to compile the lnd proto +file in Python before you can use it to communicate with lnd. + +* Create a virtual environment for your project +``` +$ virtualenv lnd +``` +* Activate the virtual environment +``` +$ source lnd/bin/activate +``` +* Install dependencies (googleapis-common-protos is required due to the use of + google/api/annotations.proto) +``` +(lnd)$ pip install grpcio grpcio-tools googleapis-common-protos +``` +* Clone the google api's repository (required due to the use of + google/api/annotations.proto) +``` +(lnd)$ git clone https://github.com/googleapis/googleapis.git +``` +* Copy the lnd rpc.proto file (you'll find this at + [lnrpc/rpc.proto](https://github.com/lightningnetwork/lnd/blob/master/lnrpc/rpc.proto)) + or just download it +``` +(lnd)$ curl -o rpc.proto -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/lnrpc/rpc.proto +``` +* Compile the proto file +``` +(lnd)$ python -m grpc_tools.protoc --proto_path=googleapis:. --python_out=. --grpc_python_out=. rpc.proto +``` + +After following these steps, two files `rpc_pb2.py` and `rpc_pb2_grpc.py` will +be generated. These files will be imported in your project anytime you use +Python gRPC. + +#### Imports and Client + +Every time you use Python gRPC, you will have to import the generated rpc modules +and set up a channel and stub to your connect to your `lnd` node: + +```python +import rpc_pb2 as ln +import rpc_pb2_grpc as lnrpc +import grpc +import os + +# Due to updated ECDSA generated tls.cert we need to let gprc know that +# we need to use that cipher suite otherwise there will be a handhsake +# error when we communicate with the lnd rpc server. +os.environ["GRPC_SSL_CIPHER_SUITES"] = 'HIGH+ECDSA' + +# Lnd cert is at ~/.lnd/tls.cert on Linux and +# ~/Library/Application Support/Lnd/tls.cert on Mac +cert = open(os.path.expanduser('~/.lnd/tls.cert'), 'rb').read() +creds = grpc.ssl_channel_credentials(cert) +channel = grpc.secure_channel('localhost:10009', creds) +stub = lnrpc.LightningStub(channel) +``` + +### Examples + +Let's walk through some examples of Python gRPC clients. These examples assume +that you have at least two `lnd` nodes running, the RPC location of one of which +is at the default `localhost:10009`, with an open channel between the two nodes. + +#### Simple RPC + +```python +# Retrieve and display the wallet balance +response = stub.WalletBalance(ln.WalletBalanceRequest()) +print(response.total_balance) +``` + +#### Response-streaming RPC + +```python +request = ln.InvoiceSubscription() +for invoice in stub.SubscribeInvoices(request): + print(invoice) +``` + +Now, create an invoice for your node at `localhost:10009`and send a payment to +it from another node. +```bash +$ lncli addinvoice --amt=100 +{ + "r_hash": , + "pay_req": +} +$ lncli sendpayment --pay_req= +``` + +Your Python console should now display the details of the recently satisfied +invoice. + +#### Bidirectional-streaming RPC + +```python +from time import sleep +import codecs + +def request_generator(dest, amt): + # Initialization code here + counter = 0 + print("Starting up") + while True: + request = ln.SendRequest( + dest=dest, + amt=amt, + ) + yield request + # Alter parameters here + counter += 1 + sleep(2) + +# Outputs from lncli are hex-encoded +dest_hex = +dest_bytes = codecs.decode(dest_hex, 'hex') + +request_iterable = request_generator(dest=dest_bytes, amt=100) + +for payment in stub.SendPayment(request_iterable): + print(payment) +``` +This example will send a payment of 100 satoshis every 2 seconds. + +#### Using Macaroons + +To authenticate using macaroons you need to include the macaroon in the metadata of the request. + +```python +import codecs + +# Lnd admin macaroon is at ~/.lnd/data/chain/bitcoin/simnet/admin.macaroon on Linux and +# ~/Library/Application Support/Lnd/data/chain/bitcoin/simnet/admin.macaroon on Mac +with open(os.path.expanduser('~/.lnd/data/chain/bitcoin/simnet/admin.macaroon'), 'rb') as f: + macaroon_bytes = f.read() + macaroon = codecs.encode(macaroon_bytes, 'hex') +``` + +The simplest approach to use the macaroon is to include the metadata in each request as shown below. + +```python +stub.GetInfo(ln.GetInfoRequest(), metadata=[('macaroon', macaroon)]) +``` + +However, this can get tiresome to do for each request, so to avoid explicitly including the macaroon we can update the credentials to include it automatically. + +```python +def metadata_callback(context, callback): + # for more info see grpc docs + callback([('macaroon', macaroon)], None) + + +# build ssl credentials using the cert the same as before +cert_creds = grpc.ssl_channel_credentials(cert) + +# now build meta data credentials +auth_creds = grpc.metadata_call_credentials(metadata_callback) + +# combine the cert credentials and the macaroon auth credentials +# such that every call is properly encrypted and authenticated +combined_creds = grpc.composite_channel_credentials(cert_creds, auth_creds) + +# finally pass in the combined credentials when creating a channel +channel = grpc.secure_channel('localhost:10009', combined_creds) +stub = lnrpc.LightningStub(channel) + +# now every call will be made with the macaroon already included +stub.GetInfo(ln.GetInfoRequest()) +``` + + +### Conclusion + +With the above, you should have all the `lnd` related `gRPC` dependencies +installed locally into your virtual environment. In order to get up to speed +with `protobuf` usage from Python, see [this official `protobuf` tutorial for +Python](https://developers.google.com/protocol-buffers/docs/pythontutorial). +Additionally, [this official gRPC +resource](http://www.grpc.io/docs/tutorials/basic/python.html) provides more +details around how to drive `gRPC` from Python. + + diff --git a/render.py b/render.py new file mode 100755 index 0000000..ab20ab0 --- /dev/null +++ b/render.py @@ -0,0 +1,87 @@ +#!/usr/bin/python2.7 +from jinja2 import Environment, FileSystemLoader, select_autoescape + + +def read_file_without_title(filename): + """ + Takes filename of markdown as input and returns the file contents as a + utf-8 encoded string after stripping out the title + """ + file_lines = open(filename).readlines() + + while True: + line = file_lines[0].decode('utf-8').strip() + + # If the line is empty or begins with a `# ` denoting a h1 md header + if line == '' or line[:2] == '# ': + # Remove the first line + file_lines.pop(0) + else: + break + + # At this point, file_lines is stripped of the title. + # Return the utf string + return "".join(file_lines) + + +def render(): + """ + Given the necessary markdown files that are maintained on Github in the + `lnd` repo, renders the guides with Jekyll header. + """ + + # Load the Jekyll header from the `templates` dir + env = Environment( + loader=FileSystemLoader('./templates'), + autoescape=select_autoescape(['html', 'xml']) + ) + template = env.get_template('base.md') + + # Read INSTALL.md and output guides/installation.md + installation_guide = template.render( + title='Installation', + permalink=None, + content=read_file_without_title('INSTALL.md'), + footer=open('templates/installation_footer.md').read(), + ).encode('utf-8') + installation_output = 'guides/installation.md' + with open(installation_output, "wb") as file_out: + file_out.write(installation_guide) + print "Rendered {}".format(installation_output) + + # Read DOCKER-README.md and output guides/docker.md + docker_guide = template.render( + title='Working with LND and Docker', + permalink=None, + content=read_file_without_title('DOCKER-README.md'), + ).encode('utf-8') + docker_output = 'guides/docker.md' + with open(docker_output, "wb") as file_out: + file_out.write(docker_guide) + print "Rendered {}".format(docker_output) + + # Read python.md and output guides/python-grpc.md + python_grpc_guide = template.render( + title='How to write a Python gRPC client for the Lightning Network Daemon', + permalink=None, + content=read_file_without_title('python.md'), + ).encode('utf-8') + python_grpc_output = 'guides/python-grpc.md' + with open(python_grpc_output, "wb") as file_out: + file_out.write(python_grpc_guide) + print "Rendered {}".format(python_grpc_output) + + # Read javascript.md and output guides/javascript-grpc.md + javascript_grpc_guide = template.render( + title='How to write a Javascript gRPC client for the Lightning Network Daemon', + permalink=None, + content=read_file_without_title('javascript.md'), + ).encode('utf-8') + javascript_grpc_output = 'guides/javascript-grpc.md' + with open(javascript_grpc_output, "wb") as file_out: + file_out.write(javascript_grpc_guide) + print "Rendered {}".format(javascript_grpc_output) + + +if __name__ == '__main__': + render() diff --git a/s3_website.yml b/s3_website.yml deleted file mode 100644 index fc8a742..0000000 --- a/s3_website.yml +++ /dev/null @@ -1,65 +0,0 @@ -s3_id: <%= ENV['LN_S3_ID'] %> -s3_secret: <%= ENV['LN_S3_SECRET'] %> -s3_bucket: dev.lightning.community -cloudfront_distribution_id: <%= ENV['LN_CLOUDFRONT_DISTRIBUTION_ID'] %> - -# set s3_token if using temporary credentials with a session token (eg: when assuming a role) -# s3_token: YOUR_AWS_S3_SESSION_TOKEN - -# Below are examples of all the available configurations. -# See README for more detailed info on each of them. - -# site: path-to-your-website - -# index_document: index.html -# error_document: error.html - -# max_age: -# "assets/*": 6000 -# "*": 300 - -# gzip: -# - .html -# - .css -# - .md -# gzip_zopfli: true - -# See http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region for valid endpoints -# s3_endpoint: ap-northeast-1 - -# ignore_on_server: that_folder_of_stuff_i_dont_keep_locally - -# exclude_from_upload: -# - those_folders_of_stuff -# - i_wouldnt_want_to_upload - -# s3_reduced_redundancy: true - -# cloudfront_distribution_id: your-dist-id - -# cloudfront_distribution_config: -# default_cache_behavior: -# min_ttl: <%= 60 * 60 * 24 %> -# aliases: -# quantity: 1 -# items: -# - your.website.com - -# cloudfront_invalidate_root: true - -cloudfront_wildcard_invalidation: true - -# concurrency_level: 5 - -# redirects: -# index.php: / -# about.php: about.html -# music-files/promo.mp4: http://www.youtube.com/watch?v=dQw4w9WgXcQ - -# routing_rules: -# - condition: -# key_prefix_equals: blog/some_path -# redirect: -# host_name: blog.example.com -# replace_key_prefix_with: some_new_path/ -# http_redirect_code: 301 diff --git a/templates/base.md b/templates/base.md new file mode 100644 index 0000000..b12f375 --- /dev/null +++ b/templates/base.md @@ -0,0 +1,11 @@ +--- +layout: page +title: {{ title }} +{% if permalink %}permalink: {{ permalink }} {% endif %} +--- + +{{ content }} + +{% if footer %} +{{ footer }} +{% endif %} diff --git a/templates/installation_footer.md b/templates/installation_footer.md new file mode 100644 index 0000000..225a142 --- /dev/null +++ b/templates/installation_footer.md @@ -0,0 +1,8 @@ + +### Next Steps + +* **[Tutorial](/tutorial/):** Get acquainted with the skills necessary for `lnd` development. +* **[Developer Guides](/guides/):** Look through developer manuals on gRPC, + Docker, and more. +* **[Resources](/resources/):** Learn about the Lightning Network +* **[Code Contribution Guidelines](/contribute/):** Contribute to `lnd` itself. diff --git a/tutorial/01-lncli.md b/tutorial/01-lncli.md index abed680..22d21c6 100644 --- a/tutorial/01-lncli.md +++ b/tutorial/01-lncli.md @@ -1,8 +1,7 @@ --- layout: page title: Stage 1 - Setting up a local cluster ---- - +--- ### Introduction In this stage of the tutorial, we will learn how to set up a local cluster of @@ -12,8 +11,10 @@ baseline understanding of the different components that must work together as part of developing on `lnd`. This tutorial assumes you have completed installation of Go, `btcd`, and `lnd` -on simnet. If not, you can find the installation instructions -[here](/installation/). +on simnet. If not, please refer to the [installation +instructions](/guides/installation/). Note that for the purposes of this +tutorial it is not necessary to sync testnet, and the last section of the +installation instructions you'll need to complete is "Installing btcd." The schema will be the following. Keep in mind that you can easily extend this network to include additional nodes `David`, `Eve`, etc. by simply running more @@ -23,15 +24,15 @@ local `lnd` instances. ``` (1) (1) (1) + ----- + + --- + + ------- + -| Alice | <--- channel ---> | Bob | <--- channel ---> | Charlie | -+ ----- + + --- + + ------- + - | | | +| Alice | <--- channel ---> | Bob | <--- channel ---> | Charlie | ++ ----- + + --- + + ------- + | | | - + - - - - - - - - - - - - + - - - - - - - - - - - - - + + | | | + + - - - - - - - - - - - - + - - - - - - - - - - - - - + | + --------------- + | BTC/LTC network | <--- (2) - + --------------- + + + --------------- + ``` ### Understanding the components @@ -46,7 +47,10 @@ from the underlying Bitcoin network itself. Running an `lnd` node means that it is listening for payments, watching the blockchain, etc. By default it is awaiting user input. -`lncli` is the command line client used to interact with your `lnd` nodes. Typically, each `lnd` node will be running in its own terminal window, so that you can see its log outputs. `lncli` commands are thus run from a different terminal window. +`lncli` is the command line client used to interact with your `lnd` nodes. +Typically, each `lnd` node will be running in its own terminal window, so that +you can see its log outputs. `lncli` commands are thus run from a different +terminal window. #### BTCD @@ -60,7 +64,7 @@ will be connected to their own instances of `btcd` or equivalent. We will also be using `simnet` instead of `testnet`. Simnet is a development/test network run locally that allows us to generate blocks at will, so we can avoid the time-consuming process of waiting for blocks to arrive for -any on-chain (typically channel-related) functionality. +any on-chain functionality. ### Setting up our environment @@ -73,19 +77,22 @@ workflow. Let's start by running btcd, if you don't have it up already. Open up a new terminal window, ensure you have your `$GOPATH` set, and run: ```bash -btcd --simnet --txindex --rpcuser=kek --rpcpass=kek +btcd --txindex --simnet --rpcuser=kek --rpcpass=kek ``` +(Note: this tutorial requires opening quite a few terminal windows. It may be +convenient to use multiplexers such as tmux or screen if you're familiar with +them.) Breaking down the components: + * `--txindex` is required so that the `lnd` client is able to query + historical transactions from `btcd`. * `--simnet` specifies that we are using the `simnet` network. This can be changed to `--testnet`, or omitted entirely to connect to the actual Bitcoin / Litecoin network. - * `--txindex` is required so that the `lnd` client is able to query - historical transactions from `btcd`. * `--rpcuser` and `rpcpass` sets a default password for authenticating to the `btcd` instance. -#### Running lnd +#### Starting lnd (Alice's node) Now, let's set up the three `lnd` nodes. To keep things as clean and separate as possible, open up a new terminal window, ensure you have `$GOPATH` set and @@ -93,7 +100,7 @@ as possible, open up a new terminal window, ensure you have `$GOPATH` set and `dev` that will represent our development space. We will create separate folders to store the state for alice, bob, and charlie, and run all of our `lnd` nodes on different `localhost` ports instead of using -[Docker](/docker-guide/) to make our networking a bit easier. +[Docker](/guides/docker/) to make our networking a bit easier. ```bash # Create our development space @@ -102,9 +109,7 @@ mkdir dev cd dev # Create folders for each of our nodes -mkdir alice -mkdir bob -mkdir charlie +mkdir alice bob charlie ``` The directory structure should now look like this: @@ -112,33 +117,34 @@ The directory structure should now look like this: $ tree $GOPATH -L 2 ├── bin -│   └── ... +│ └── ... ├── dev -│   ├── alice -│   ├── bob -│   └── charlie +│ ├── alice +│ ├── bob +│ └── charlie ├── pkg -│   └── ... +│ └── ... ├── rpc -│   └── ... +│ └── ... └── src └── ... ``` Start up the Alice node from within the `alice` directory: ```bash -$ cd $GOPATH/dev/alice -alice$ lnd --rpcport=10001 --peerport=10011 --restport=8001 --datadir=test_data --logdir=test_log --debuglevel=info --bitcoin.rpcuser=kek --bitcoin.rpcpass=kek --bitcoin.simnet --bitcoin.active +cd $GOPATH/dev/alice +alice$ lnd --rpclisten=localhost:10001 --listen=localhost:10011 --restlisten=localhost:8001 --datadir=data --logdir=log --debuglevel=info --bitcoin.simnet --bitcoin.active --bitcoin.node=btcd --btcd.rpcuser=kek --btcd.rpcpass=kek ``` -The Alice node should now be running and displaying output. +The Alice node should now be running and displaying output ending with a line +beginning with "Waiting for wallet encryption password." Breaking down the components: - * `--rpcport`: The port to listen for the RPC server. This is the primary way + * `--rpclisten`: The host:port to listen for the RPC server. This is the primary way an application will communicate with `lnd` - * `--peerport`: The port to listen on for incoming P2P + * `--listen`: The host:port to listen on for incoming P2P connections. This is at the networking level, and is distinct from the Lightning channel networks and Bitcoin/Litcoin network itself. - * `--restport`: The port exposing a REST api for interacting with `lnd` over + * `--restlisten`: The host:port exposing a REST api for interacting with `lnd` over HTTP. For example, you can get Alice's channel balance by making a GET request to `localhost:8001/v1/channels`. This is not needed for this tutorial, but you can see some examples @@ -147,13 +153,16 @@ Breaking down the components: * `--logdir`: The directory to log output. * `--debuglevel`: The logging level for all subsystems. Can be set to `trace`, `debug`, `info`, `warn`, `error`, `critical`. - * `--bitcoin.rpcuser` and `--bitcoin.rpcpass`: The username and password for - the `btcd` instance * `--bitcoin.simnet`: Specifies whether to use `simnet` or `testnet` * `--bitcoin.active`: Specifies that bitcoin is active. Can also include - `--litecoin.active` to activate Litecoin + `--litecoin.active` to activate Litecoin. + * `--bitcoin.node=btcd`: Use the `btcd` full node to interface with the blockchain. + Note that when using Litecoin, the option is `--litecoin.node=btcd`. + * `--btcd.rpcuser` and `--btcd.rpcpass`: The username and password for + the `btcd` instance. Note that when using Litecoin, the options are `--ltcd.rpcuser` + and `--ltcd.rpcpass`. -### Running Bob and Charlie +### Starting Bob's node and Charlie's node Just as we did with Alice, start up the Bob node from within the `bob` directory, and the Charlie node from within the `charlie` directory. Doing so @@ -161,10 +170,10 @@ will configure the `datadir` and `logdir` to be in separate locations so that there is never a conflict. Keep in mind that for each additional terminal window you set, you will need to -set `$GOPATH` and include `$GOPATH/bin` in your `PATH`. You could create a +set `$GOPATH` and include `$GOPATH/bin` in your `PATH`. Consider creating a setup script that includes the following lines: ```bash -export GOPATH=~/github/ln-lnd +export GOPATH=~/gocode # if you exactly followed the install guide export PATH=$PATH:$GOPATH/bin ``` and run it every time you start a new terminal window working on `lnd`. @@ -173,11 +182,11 @@ Run Bob and Charlie: ```bash # In a new terminal window cd $GOPATH/dev/bob -bob$ lnd --rpcport=10002 --peerport=10012 --restport=8002 --datadir=test_data --logdir=test_log --debuglevel=info --bitcoin.rpcuser=kek --bitcoin.rpcpass=kek --bitcoin.simnet --bitcoin.active +bob$ lnd --rpclisten=localhost:10002 --listen=localhost:10012 --restlisten=localhost:8002 --datadir=data --logdir=log --debuglevel=info --bitcoin.simnet --bitcoin.active --bitcoin.node=btcd --btcd.rpcuser=kek --btcd.rpcpass=kek # In another terminal window cd $GOPATH/dev/charlie -charlie$ lnd --rpcport=10003 --peerport=10013 --restport=8003 --datadir=test_data --logdir=test_log --debuglevel=info --bitcoin.rpcuser=kek --bitcoin.rpcpass=kek --bitcoin.simnet --bitcoin.active +charlie$ lnd --rpclisten=localhost:10003 --listen=localhost:10013 --restlisten=localhost:8003 --datadir=data --logdir=log --debuglevel=info --bitcoin.simnet --bitcoin.active --bitcoin.node=btcd --btcd.rpcuser=kek --btcd.rpcpass=kek ``` ### Configuring lnd.conf @@ -191,59 +200,74 @@ overwrite the `lnd.conf` option if applicable. - On MacOS, `lnd.conf` is located at: `/Users/[username]/Library/Application\ Support/Lnd/lnd.conf` - On Linux: `~/.lnd/lnd.conf` -Here is an example `lnd.conf` that can save us from re-specifying a bunch of command line options: +Here is an example `lnd.conf` that can save us from re-specifying a bunch of +command line options: ```bash [Application Options] -datadir=test_data -logdir=test_log +datadir=data +logdir=log debuglevel=info -debughtlc=true [Bitcoin] -bitcoin.simnet -bitcoin.active -bitcoin.rpcuser=kek -bitcoin.rpcpass=kek +bitcoin.simnet=1 +bitcoin.active=1 +bitcoin.node=btcd + +[btcd] +btcd.rpcuser=kek +btcd.rpcpass=kek ``` Now, when we start nodes, we only have to type ```bash -alice$ lnd --rpcport=10001 --peerport=10011 --restport=8001 -bob$ lnd --rpcport=10002 --peerport=10012 --restport=8002 -charlie$ lnd --rpcport=10003 --peerport=10013 --restport=8003 +alice$ lnd --rpclisten=localhost:10001 --listen=localhost:10011 --restlisten=localhost:8001 +bob$ lnd --rpclisten=localhost:10002 --listen=localhost:10012 --restlisten=localhost:8002 +charlie$ lnd --rpclisten=localhost:10003 --listen=localhost:10013 --restlisten=localhost:8003 ``` etc. -### Working with lncli +### Working with lncli and authentication Now that we have our `lnd` nodes up and running, let's interact with them! To control `lnd` we will need to use `lncli`, the command line interface. +`lnd` uses [macaroons](https://github.com/lightningnetwork/lnd/issues/20) for +authentication to the rpc server. `lncli` typically looks for an +`admin.macaroon` file in the Lnd home directory, but since we changed the +location of our application data, we have to set `--macaroonpath` in the +following command. To disable macaroons, pass the `--no-macaroons` flag into +both `lncli` and `lnd`. + +`lnd` allows you to encrypt your wallet with a passphrase and optionally encrypt +your cipher seed passphrase as well. This can be turned off by passing +`--noencryptwallet` into `lnd` or `lnd.conf`. We recommend going through this +process at least once to familiarize yourself with the security and +authentication features around `lnd`. + +We will test our rpc connection to the Alice node. Notice that in the following command we specify the +`--rpcserver` here, which corresponds to `--rpcport=10001` that we set when +starting the Alice `lnd` node. + Open up a new terminal window, set `$GOPATH` and include `$GOPATH/bin` in your -`PATH` as usual. Let's test that we can connect to Alice by requesting basic information: +`PATH` as usual. Let's create Alice's wallet and set her passphrase: ```bash cd $GOPATH/dev/alice -alice$ lncli --rpcserver=localhost:10001 getinfo +alice$ lncli --rpcserver=localhost:10001 --macaroonpath=data/chain/bitcoin/simnet/admin.macaroon create ``` +You'll be asked to input and confirm a wallet password for Alice, which must be +longer than 8 characters. You also have the option to add a passphrase to your +cipher seed. For now, just skip this step by entering "n" when prompted about +whether you have an existing mnemonic, and pressing enter to proceed without the +passphrase. -`lncli` just made an RPC call to the Alice `lnd` node. Notice that we had to -specify the `--rpcserver` here, which corresponds to port `10001` that we set -when starting the Alice `lnd` node. - -#### lncli options - -To see all the commands available for `lncli`, simply type `lncli --help` or -`lncli -h`. - -### Setting up Bitcoin addresses -Let's create a new Bitcoin address for Alice. This will be the address that -stores Alice's on-chain balance. +You can now request some basic information as follows: ```bash -alice$ lncli --rpcserver=localhost:10001 newaddress np2wkh -{ - "address": -} +alice$ lncli --rpcserver=localhost:10001 --macaroonpath=data/chain/bitcoin/simnet/admin.macaroon getinfo ``` +`lncli` just made an RPC call to the Alice `lnd` node. This is a good way to +test if your nodes are up and running and `lncli` is functioning properly. Note +that in future sessions you may need to call `lncli unlock` to unlock the node +with the password you just set. Open up new terminal windows and do the same for Bob and Charlie. `alice$` or `bob$` denotes running the command from the Alice or Bob `lncli` window @@ -251,25 +275,21 @@ respectively. ```bash # In a new terminal window, setting $GOPATH, etc. cd $GOPATH/dev/bob -bob$ lncli --rpcserver=localhost:10002 newaddress np2wkh -{ - "address": -} +bob$ lncli --rpcserver=localhost:10002 --macaroonpath=data/chain/bitcoin/simnet/admin.macaroon create +# Note that you'll have to enter an 8+ character password and "n" for the mnemonic. # In a new terminal window: -cd $GOPATH/dev/bob -charlie$ lncli --rpcserver=localhost:10003 newaddress np2wkh -{ - "address": -} +cd $GOPATH/dev/charlie +charlie$ lncli --rpcserver=localhost:10003 --macaroonpath=data/chain/bitcoin/simnet/admin.macaroon create +# Note that you'll have to enter an 8+ character password and "n" for the mnemonic. ``` -To avoid typing the `--rpcserver=localhost:1000X` flag every time, we can set -some aliases. Add the following to your `.bashrc`: +To avoid typing the `--rpcserver=localhost:1000X` and `--macaroonpath` flag +every time, we can set some aliases. Add the following to your `.bashrc`: ```bash -alias lncli-alice="lncli --rpcserver=localhost:10001" -alias lncli-bob="lncli --rpcserver=localhost:10002" -alias lncli-charlie="lncli --rpcserver=localhost:10003" +alias lncli-alice="lncli --rpcserver=localhost:10001 --macaroonpath=data/chain/bitcoin/simnet/admin.macaroon" +alias lncli-bob="lncli --rpcserver=localhost:10002 --macaroonpath=data/chain/bitcoin/simnet/admin.macaroon" +alias lncli-charlie="lncli --rpcserver=localhost:10003 --macaroonpath=data/chain/bitcoin/simnet/admin.macaroon" ``` To make sure this was applied to all of your current terminal windows, rerun @@ -279,13 +299,44 @@ alice$ source ~/.bashrc bob$ source ~/.bashrc charlie$ source ~/.bashrc ``` -For simplicity, the rest of the tutorial will assume that this step was complete. +For simplicity, the rest of the tutorial will assume that this step was +complete. + +#### lncli options + +To see all the commands available for `lncli`, simply type `lncli --help` or +`lncli -h`. + +### Setting up Bitcoin addresses +Let's create a new Bitcoin address for Alice. This will be the address that +stores Alice's on-chain balance. `np2wkh` specifes the type of address and +stands for Pay to Nested Witness Key Hash. + +```bash +alice$ lncli-alice newaddress np2wkh +{ + "address": +} +``` + +And for Bob and Charlie: +```bash +bob$ lncli-bob newaddress np2wkh +{ + "address": +} + +charlie$ lncli-charlie newaddress np2wkh +{ + "address": +} +``` ### Funding Alice -That's a lot of configuration! Recall that at this point, we've generated -onchain addresses for Alice, Bob, and Charlie. Now, we will get some practice -working with `btcd` and fund these addresses with some `simnet` Bitcoin. +That's a lot of configuration! At this point, we've generated onchain addresses +for Alice, Bob, and Charlie. Now, we will get some practice working with `btcd` +and fund these addresses with some `simnet` Bitcoin. Quit btcd and re-run it, setting Alice as the recipient of all mining rewards: ```bash @@ -293,34 +344,33 @@ btcd --simnet --txindex --rpcuser=kek --rpcpass=kek --miningaddr= ``` Generate 400 blocks, so that Alice gets the reward. We need at least 100 blocks -because coinbase funds can't be spent until after 100 confirmations, and we -need about 300 to activate segwit. The following command can be run in any -window with `$GOPATH` and `$PATH` set. +because coinbase funds can't be spent until after 100 confirmations, and we need +about 300 to activate segwit. In a new window with `$GOPATH` and `$PATH` set: ```bash -btcctl generate 400 +alice$ btcctl --simnet --rpcuser=kek --rpcpass=kek generate 400 ``` Check that segwit is active: ```bash -btcctl getblockchaininfo | grep -A 1 segwit +btcctl --simnet --rpcuser=kek --rpcpass=kek getblockchaininfo | grep -A 1 segwit ``` -Check Alice's wallet balance. `--witness_only=true` specifies that we only want -to consider witness outputs when calculating the wallet balance. +Check Alice's wallet balance. ```bash -alice$ lncli-alice walletbalance --witness_only=true +alice$ lncli-alice walletbalance ``` -It's no fun if only Alice any money. Let's give some to Charlie as well: +It's no fun if only Alice has money. Let's give some to Charlie as well. ```bash -# Quit btcd -btcd --simnet --txindex --rpcuser=kek --rpcpass=kek --miningaddr= +# Quit the btcd process that was previously started with Alice's mining address, +# and then restart it with: +btcd --txindex --simnet --rpcuser=kek --rpcpass=kek --miningaddr= # Generate more blocks -btcctl generate 50 +btcctl --simnet --rpcuser=kek --rpcpass=kek generate 100 # Check Charlie's balance -charlie$ lncli-charlie walletbalance --witness_only=true +charlie$ lncli-charlie walletbalance ``` ### Creating the P2P Network @@ -332,25 +382,32 @@ Connect Alice to Bob: # Get Bob's identity pubkey: bob$ lncli-bob getinfo { - ----->"identity_pubkey": , +--->"identity_pubkey": , "alias": "", "num_pending_channels": 0, "num_active_channels": 0, "num_peers": 0, - "block_height": 1215, - "block_hash": "7d0bc86ea4151ed3b5be908ea883d2ac3073263537bcf8ca2dca4bec22e79d50", + "block_height": 450, + "block_hash": "2a84b7a2c3be81536ef92cf382e37ab77f7cfbcf229f7d553bb2abff3e86231c", "synced_to_chain": true, - "testnet": false + "testnet": false, "chains": [ "bitcoin" - ] + ], + "uris": [ + ], + "best_header_timestamp": "1533350134", + "version": "0.4.2-beta commit=7a5a824d179c6ef16bd78bcb7a4763fda5f3f498" } # Connect Alice and Bob together alice$ lncli-alice connect @localhost:10012 +{ + +} ``` -Notice that `localhost:10012` corresponds to the `--peerport=10012` flag we set when -starting the Bob `lnd` node. +Notice that `localhost:10012` corresponds to the `--listen=localhost:10012` flag we set +when starting the Bob `lnd` node. Let's check that Alice and Bob are now aware of each other. ```bash @@ -360,13 +417,12 @@ alice$ lncli-alice listpeers "peers": [ { "pub_key": , - "peer_id": 1, - "address": "172.19.0.4:5656", - "bytes_sent": "357", - "bytes_recv": "357", + "address": "127.0.0.1:10012", + "bytes_sent": "7", + "bytes_recv": "7", "sat_sent": "0", "sat_recv": "0", - "inbound": true, + "inbound": false, "ping_time": "0" } ] @@ -378,14 +434,13 @@ bob$ lncli-bob listpeers "peers": [ { "pub_key": , - "peer_id": 1, - "address": "172.19.0.3:51932", - "bytes_sent": "357", - "bytes_recv": "357", + "address": "127.0.0.1:60104", + "bytes_sent": "318", + "bytes_recv": "318", "sat_sent": "0", "sat_recv": "0", - "inbound": false, - "ping_time": "0" + "inbound": true, + "ping_time": "5788" } ] } @@ -403,16 +458,15 @@ to Bob, and Bob to Charlie. First, let's open the Alice<-->Bob channel. ```bash -alice$ lncli-alice openchannel --node_key= --num_confs=1 --local_amt=1000000 +alice$ lncli-alice openchannel --node_key= --local_amt=1000000 ``` -- `--num_confs` specifies that we will only wait one confirmation before we -consider the channel to be 'open', and therefore functional. -- `--local_amt` specifies the amount of money that Alice will commit to the channel. -To see the full list of options, you can try `lncli openchannel --help`. +- `--local_amt` specifies the amount of money that Alice will commit to the + channel. To see the full list of options, you can try `lncli openchannel + --help`. -We now need to mine one block so that the channel is considered valid: +We now need to mine six blocks so that the channel is considered valid: ```bash -btcctl generate 1 +btcctl --simnet --rpcuser=kek --rpcpass=kek generate 6 ``` Check that Alice<-->Bob channel was created: @@ -423,19 +477,26 @@ alice$ lncli-alice listchannels { "active": true, "remote_pubkey": , - "channel_point": "3511ae8a52c97d957eaf65f828504e68d0991f0276adff94c6ba91c7f6cd4275:0", - "chan_id": "1337006139441152", - "capacity": "1005000", - "local_balance": "1000000", + "channel_point": "2622b779a8acca471a738b0796cd62e4457b79b33265cbfa687aadccc329023a:0", + "chan_id": "495879744192512", + "capacity": "1000000", + "local_balance": "991312", "remote_balance": "0", + "commit_fee": "8688", + "commit_weight": "600", + "fee_per_kw": "12000", "unsettled_balance": "0", "total_satoshis_sent": "0", "total_satoshis_received": "0", - "num_updates": "0" + "num_updates": "0", + "pending_htlcs": [ + ], + "csv_delay": 144, + "private": false } ] } -``` +``` ### Sending single hop payments @@ -444,16 +505,35 @@ Alice to Bob. First, Bob will need to generate an invoice: ```bash -bob$ lncli-bob addinvoice --value=10000 +bob$ lncli-bob addinvoice --amt=10000 { - "r_hash": "", - "pay_req": "", + "r_hash": "", + "pay_req": "", + "add_index": 1 } ``` Send the payment from Alice to Bob: ```bash alice$ lncli-alice sendpayment --pay_req= +{ + "payment_error": "", + "payment_preimage": "baf6929fc95b3824fb774a4b75f6c8a1ad3aaef04efbf26cc064904729a21e28", + "payment_route": { + "total_time_lock": 650, + "total_amt": 10000, + "hops": [ + { + "chan_id": 495879744192512, + "chan_capacity": 1000000, + "amt_to_forward": 10000, + "expiry": 650, + "amt_to_forward_msat": 10000000 + } + ], + "total_amt_msat": 10000000 + } +} # Check that Alice's channel balance was decremented accordingly: alice$ lncli-alice listchannels @@ -464,23 +544,32 @@ bob$ lncli-bob listchannels ### Multi-hop payments -Now that we know how to send single-hop payments, sending multi hop payments is not that much more difficult. Let's set up a channel from Bob<-->Charlie: +Now that we know how to send single-hop payments, sending multi hop payments is +not that much more difficult. Let's set up a channel from Bob<-->Charlie: ```bash -charlie$ lncli-charlie openchannel --node_key= --num_confs=1 --local_amt=1000000 +charlie$ lncli-charlie openchannel --node_key= --local_amt=800000 --push_amt=200000 + +# Mine the channel funding tx +btcctl --simnet --rpcuser=kek --rpcpass=kek generate 6 ``` +Note that this time, we supplied the `--push_amt` argument, which specifies the +amount of money we want the other party to have at the first channel state. + Let's make a payment from Alice to Charlie by routing through Bob: ```bash -charlie$ lncli-charlie addinvoice --value=10000 +charlie$ lncli-charlie addinvoice --amt=10000 alice$ lncli-alice sendpayment --pay_req= -# Check that Charlie's channel was credited with the payment amount: +# Check that Charlie's channel was credited with the payment amount (e.g. that +# the `remote_balance` has been decremented by 10000) charlie$ lncli-charlie listchannels ``` ### Closing channels -For practice, let's try closing a channel. You can always reopen it again later. +For practice, let's try closing a channel. We will reopen it in the next stage +of the tutorial. ```bash alice$ lncli-alice listchannels @@ -508,32 +597,40 @@ identifies the channel. The first number is `funding_txid` and the second number is `output_index`. ```bash # Close the Alice<-->Bob channel from Alice's side. -alice$ lncli closechannel --funding_txid= --output_index= +alice$ lncli-alice closechannel --funding_txid= --output_index= # Mine a block including the channel close transaction to close the channel: -btcctl generate 1 +btcctl --simnet --rpcuser=kek --rpcpass=kek generate 1 # Check that Bob's on-chain balance was credited by his settled amount in the # channel. Recall that Bob previously had no on-chain Bitcoin: -alice$ lncli-alice walletbalance +bob$ lncli-bob walletbalance { - "balance": 0.0001 + "total_balance": "20001", + "confirmed_balance": "20001", + "unconfirmed_balance": "0" } ``` At this point, you've learned how to work with `btcd`, `btcctl`, `lnd`, and -`lncli`. In Stage 2, we will learn how to set up and interact with `lnd` using -a web GUI client. Click [here](/tutorial/02-web-client) to proceed. +`lncli`. In [Stage 2](/tutorial/02-web-client), we will learn how to set up and +interact with `lnd` using a web GUI client. _In the future, you can try running through this workflow on `testnet` instead of `simnet`, where you can interact with and send payments through the testnet Lightning Faucet node. For more information, see the "Connect to faucet -lightning node" section in the [Docker guide](/docker-guide/) or check out the +lightning node" section in the [Docker guide](/guides/docker/) or check out the [Lightning Network faucet repository](https://github.com/lightninglabs/lightning-faucet)._ [//]: # (TODO Max: Replace the link to Github LN Faucet to an internal guide for interacting with the Lightning Faucet) +#### Navigation +- [Proceed to Stage 2 - Web Client](/tutorial/02-web-client) +- [Return to main tutorial page](/tutorial/) + ### Questions -[![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)] -(https://webchat.freenode.net/?channels=lnd) +- Join the #dev-help channel on our [Community + Slack](https://join.slack.com/t/lightningcommunity/shared_invite/enQtMzQ0OTQyNjE5NjU1LWRiMGNmOTZiNzU0MTVmYzc1ZGFkZTUyNzUwOGJjMjYwNWRkNWQzZWE3MTkwZjdjZGE5ZGNiNGVkMzI2MDU4ZTE) +- Join IRC: + [![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)](https://webchat.freenode.net/?channels=lnd) diff --git a/tutorial/02-web-client.md b/tutorial/02-web-client.md new file mode 100644 index 0000000..f963ada --- /dev/null +++ b/tutorial/02-web-client.md @@ -0,0 +1,72 @@ +--- +layout: page +title: Stage 2 - Web Client +--- + +In this section we will learn how to set up and interact with `lnd` using a +[web client](https://github.com/mably/lncli-web) built by [Francois +Mably](https://github.com/mably). Setup instructions are courtesy of the +creator himself. Before beginning this step make sure you have `node` and `npm` +[installed](https://nodejs.org/en/download/). + +### Setting up the LND web client + +```bash +# Clone the repo and move into it: +git clone https://github.com/mably/lncli-web $GOPATH/src/github.com/mably/lncli-web +cd $GOPATH/src/github.com/mably/lncli-web + +# Install dependencies +npm install + +# Setup default configuration files +"./node_modules/.bin/gulp" bundle + +# Setup cert file +# Enter the Lnd home directory, located by default at ~/.lnd on Linux or +# /Users/[username]/Library/Application Support/Lnd/ on Mac OSX +# $APPDATA/Local/Lnd on Windows. Also change '/CN=localhost/O=lnd' to '//CN=localhost\O=lnd' if you are using Git Bash. +cd ~/.lnd +openssl ecparam -genkey -name prime256v1 -out tls.key +openssl req -new -sha256 -key tls.key -out csr.csr -subj '/CN=localhost/O=lnd' +openssl req -x509 -sha256 -days 36500 -key tls.key -in csr.csr -out tls.cert +rm csr.csr +cp tls.cert $GOPATH/src/github.com/mably/lncli-web/lnd.cert + +# Start the server to point to our Alice node: +cd $GOPATH/src/github.com/mably/lncli-web +node server --lndhost=localhost:10001 + +# Check out the available command line arguments +node server --help +``` + +Open up [`http://localhost:8280/`](http://localhost:8280/) in your browser to see the web dashboard. + +### Poking around + +Now would be a good time to reopen that channel we had between Alice and Bob. +Except this time, we're going to do it through the web dashboard. Feel free to +try this on your own - the web dashboard is intuitive enough that we don't need +step by step instructions for it. + +Note: The Lightning desktop app is also available and is another good way to +start experimenting with Lightning. You can find it +[here](https://github.com/lightninglabs/lightning-app/releases). + +### Moving on to Step 3 + +By now, you should have gained familiarity with the web client. +In [Stage 3](/tutorial/03-rpc-client), we will learn how to set up a gRPC +client for programmatic access to our `lnd` nodes. + +#### Navigation +- [Proceed to Stage 3 - RPC Client](/tutorial/03-rpc-client) +- [Return to Stage 1 - Setting up a local cluster](/tutorial/01-lncli) +- [Return to main tutorial page](/tutorial/) + +### Questions +- Join the #dev-help channel on our [Community + Slack](https://join.slack.com/t/lightningcommunity/shared_invite/enQtMzQ0OTQyNjE5NjU1LWRiMGNmOTZiNzU0MTVmYzc1ZGFkZTUyNzUwOGJjMjYwNWRkNWQzZWE3MTkwZjdjZGE5ZGNiNGVkMzI2MDU4ZTE) +- Join IRC: + [![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)](https://webchat.freenode.net/?channels=lnd) diff --git a/tutorial/03-rpc-client.md b/tutorial/03-rpc-client.md new file mode 100644 index 0000000..0a2f967 --- /dev/null +++ b/tutorial/03-rpc-client.md @@ -0,0 +1,205 @@ +--- +layout: page +title: Stage 3 - RPC Client +--- + +In this section, we will run the finished tutorial app and set up a gRPC client +for programmatic access to our `lnd` nodes. + +### Setting up our web development workspace + +Let's set up our news site. Before beginning, ensure that you are running +Python 2.7 and that you have +[`pip`](https://pip.pypa.io/en/stable/installing/) and +[`virtualenv`](https://virtualenv.pypa.io/en/stable/installation/) installed. +```bash +# Create a new workspace which will hold both the repo and the virtualenv. We +# recommend running this in a new terminal window. +mkdir ln-workspace +cd ln-workspace + +# Clone the repo +git clone https://github.com/MaxFangX/lightning-coindesk + +# Create virtualenv and activate it. Make sure to activate this environment +# whenever you are working with the coindesk app. +virtualenv deskenv +source deskenv/bin/activate + +# Install webapp Python requirements +cd lightning-coindesk +pip install -r requirements.txt +``` + +Now, let's set up Python gRPC so that we can run our website +```bash +# Install the dependencies required for gRPC +pip install grpcio grpcio-tools googleapis-common-protos + +# Run our webserver +python manage.py runserver +``` + +Notice that in `coindesk/settings.py` we have set +`LND_RPCHOST = "localhost:10002"`. In other words, your server is connected to +the "Bob" `lnd` node, and making payments to it is equivalent to making +payments to Bob. + +### Testing the app + +If everything went smoothly, you should now have a web server exposed at port +8000. +![lightning coindesk homepage](http://i.imgur.com/D2LgBUi.png) + +Feel free to click around the site. You will soon find that you need to log in +and make a payment to view the articles. + +![login page](http://i.imgur.com/zLTEYfk.png) + +To prevent `lnd` users from having to go through the hassle of signing up with +an email address and password, we created an authentication scheme based on the +user's `lnd` identity pubkey and logging in by signing an arbitrary message. In +particular, we are signing the CSRF token sent along with the login POST +request. This scheme is secure against replay attacks because Django generates +a unique CSRF token for every login attempt, and never reuses CSRF tokens. + +Let's create a new account for Alice by logging in and supplying a username. +Copy down the generated message (in the screenshot, it is `VcccAuMC...`) + +```bash +alice$ lncli-alice signmessage +{ + "signature": +} +``` + +Paste `` into the corresponding field and set `alice` as the desired +username. You should now be logged in as Alice. + +![logged in homepage](http://imgur.com/BGSunzy.png) + +Navigate to the sweet article and make a payment from the Alice node (you can +use either the command line or the web GUI). Clicking "Complete" will prompt the +webserver to check that the payment has been complete, and you can now enjoy +reading through this sweet article. + +### Setting up gRPC + +Let's practice running some commands on gRPC. Open up a new terminal window then proceed as follows: + +```bash +# Enter the development environment +cd ln-workspace + +# Activate Python virtualenv +source deskenv/bin/activate + +# Clone the Google API repository, which is required due to the use of +# google/api/annotations.proto +git clone https://github.com/googleapis/googleapis.git + +# Download the lnd rpc.proto file +curl -o rpc.proto -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/lnrpc/rpc.proto + +# Compile the proto file +python -m grpc_tools.protoc --proto_path=googleapis:. --python_out=. --grpc_python_out=. rpc.proto +``` + +We have now generated the two files `rpc_pb2.py` and `rpc_pb2_grpc.py`, which +you'll need to import into your Python code in order to use it with lnd. Let's move it into the coindesk folder for easy access +```bash +mv rpc* lightning-coindesk/coindesk +``` + +We will now try a few commands with the Python gRPC client from the command +line. +```bash +# Optionally install ipython for prettier command line output +pip install ipython + +# Open the Django shell. This is a standard Python shell that also allows access to Django objects. +cd lightning-coindesk +python manage.py shell +``` + +```python +# Import rpc files and grpc +In [1]: from coindesk import rpc_pb2 as ln, rpc_pb2_grpc as lnrpc +In [2]: import grpc, os + +# Establish a secure connection with our RPC server. We will first have to +# gather our cert. Lnd cert is at ~/.lnd/tls.cert on Linux and +# ~/Library/Application Support/Lnd/tls.cert on Mac +In [3]: cert = open(os.path.expanduser('~/.lnd/tls.cert')).read() +In [4]: creds = grpc.ssl_channel_credentials(cert) +In [5]: channel = grpc.secure_channel('localhost:10009', creds) +# Create a new 'stub' object that will allow us to interact with our "Bob" lnd node. +In [6]: stub = lnrpc.LightningStub(channel) + +# Make a call to the ListChannels API. +In [7]: listchannels_resp = stub.ListChannels(ln.ListChannelsRequest()) +Out[7]: +channels { + active: true + remote_pubkey: "02244b8eff01be9f7b4ec1d73ab10fc36da48b01a685ac90ed09a63fe94ec08d0a" + channel_point: "2622b779a8acca471a738b0796cd62e4457b79b33265cbfa687aadccc329023a:0" + chan_id: 495879744192512 + capacity: 1000000 + local_balance: 21001 + remote_balance: 970311 + commit_fee: 8688 + commit_weight: 724 + fee_per_kw: 12000 + total_satoshis_received: 21001 + num_updates: 8 +} +channels { + active: true + remote_pubkey: "032eed260ef71110a02a5da44d82fef9628ffa51113a2d0b9524e7d3bff615a1cf" + channel_point: "028088c354b26c33cfd5a5b2d4cca27c6e3a73b6752b5beff6f67ce779af5656:1" + chan_id: 554153860464641 + capacity: 800000 + local_balance: 190000 + remote_balance: 601312 + commit_fee: 8688 + commit_weight: 724 + fee_per_kw: 12000 + total_satoshis_sent: 10000 + num_updates: 2 +} +``` +What happened here? We constructed the request object for the list channels +command with `ln.ListChannelsRequest()`, and passed it into the `ListChannels` +function exposed by our stub. + +The response was saved into a `listchannels_resp` variable that holds all the +information returned by the `listchannels` command. You can now access the +individual properties of this object; for example, you can access the `chan_id` +of Bob's first channel with `listchannels_resp.channels[0].chan_id`. + +### Playing with gRPC + +To get a hang of the RPC documentation, try running a slightly more complex +command like `GetChanInfo`, which requires passing in some parameters. The API +docs are not yet complete, but you can refer to the [RPC documentation in +lnd](https://github.com/lightningnetwork/lnd/blob/master/lnrpc/rpc.proto). + +Hint: look at the `GetChanInfo` object and the associated requests and +responses. + +### Moving on to Step 4 + +By now, you should have at least a basic understanding of how to work with lnd +from a gRPC client. In [Stage 4](/tutorial/04-webapp-integration), we will +finally integrate `lnd` into our news site. + +#### Navigation +- [Proceed to Stage 4 - Webapp Integration](/tutorial/04-webapp-integration) +- [Return to Stage 2 - Web Client](/tutorial/02-web-client) +- [Return to main tutorial page](/tutorial/) + +### Questions +- Join the #dev-help channel on our [Community + Slack](https://join.slack.com/t/lightningcommunity/shared_invite/enQtMzQ0OTQyNjE5NjU1LWRiMGNmOTZiNzU0MTVmYzc1ZGFkZTUyNzUwOGJjMjYwNWRkNWQzZWE3MTkwZjdjZGE5ZGNiNGVkMzI2MDU4ZTE) +- Join IRC: + [![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)](https://webchat.freenode.net/?channels=lnd) diff --git a/tutorial/04-webapp-integration.md b/tutorial/04-webapp-integration.md new file mode 100644 index 0000000..761174a --- /dev/null +++ b/tutorial/04-webapp-integration.md @@ -0,0 +1,414 @@ +--- +layout: page +title: Stage 4 - Webapp Integration +--- + +In this stage, we will integrate Lightning micropayments into our server-side +web application. + +### Starting Afresh + +It's time to recreate the micropayments ourselves. Get the starter code: + +```bash +# Clear any local changes first and checkout the start tag +git checkout -- . +git checkout start +``` + +The starter code has stripped out all micropayments code, and we just also just +reset our database state to the default. + +### The models + +First, let's examine `coindesk/models.py` to understand our web app a little +better. + +`coindesk/models.py` (cleaned up a little bit): +{% raw %} +```python +class Profile(models.Model): + user = models.OneToOneField(User) + created_at = models.DateTimeField(auto_now_add=True) + modified_at = models.DateTimeField(auto_now=True) + identity_pubkey = models.CharField(max_length=80, unique=True) + # bitcoin_address = BitcoinAddressField() + + +class Article(models.Model): + + ARTICLE_STATUS_CHOICES = ( + ('visible', 'Visible'), + ('deleted by admin', 'Deleted by admin'), + ) + + title = models.CharField(max_length=191) + text = models.TextField() + created_at = models.DateTimeField(auto_now_add=True) + status = models.CharField(max_length=50, default='visible', choices=ARTICLE_STATUS_CHOICES) + + @property + def views(self): + return self.payments.filter(status='complete', purpose='view').count() + + +class Payment(models.Model): + + PAYMENT_STATUS_CHOICES = ( + ('pending_invoice', 'Pending Invoice'), # Should be atomic + ('pending_payment', 'Pending Payment'), + ('complete', 'Complete'), + ('error', 'Error'), + ) + + PAYMENT_PURPOSE_CHOICES = ( + ('view', 'View'), + ('upvote', 'Upvote') + ) + + user = models.ForeignKey(User) + article = models.ForeignKey(Article, related_name='payments') + purpose = models.CharField(max_length=10, choices=PAYMENT_PURPOSE_CHOICES) + + satoshi_amount = models.IntegerField() + r_hash = models.CharField(max_length=64) + payment_request = models.CharField(max_length=1000) + + status = models.CharField(max_length=50, default='pending_invoice', choices=PAYMENT_STATUS_CHOICES) + created_at = models.DateTimeField(auto_now_add=True) + modified_at = models.DateTimeField(auto_now=True) + + def generate_invoice(self, user, article): + """ + Generates a new invoice + """ + pass + + def check_payment(self): + """ + Checks if the Lightning payment has been received for this invoice + """ + pass +``` +{% endraw %} + +Each `Profile` object is associated with one `User` object, and stores the +identity pubkey of that user. + +The `Article` class has a title, text, a timestamp and some basic moderation +functionality. Notice that the number of views for an article as counted as the +number of completed payments associated with it. + +The `Payment` class represents a Lightning micropayment, and is going to be +where we implement the bulk of our work. Each payment is associated with a particular user and a article. + +We have the `satoshi_amount`, `r_hash` and `payment_request` fields - the basic technical information +required to conduct a Lightning payment. + +Each `Payment` goes through the +following lifecycle, represented by the `status` field: + +1. The payment is initially pending an invoice. Calling `generate_invoice` will +generate an invoice for a user and a particular article they want to view. +2. After the invoice is generated, it is now pending payment. +3. `check_payment` will check if a payment has indeed been received by Bob +node. Afterwards, this `Payment` counts as complete, and the user should be +able to view the article. + +It will soon be up to us to implement the `generate_invoice` and +`check_payment` functions. But first, let's add in some security. + + +### The views + +We also need to understand some of the wiring behind the app. + +Open `coindesk/views.py`. For our purposes, the `article` view is the most +important since it is executed when we view an article and handles what to do +with the different payment states. + +{% raw %} +```python +def article(request, pk): + + try: + article = Article.objects.filter(status='visible').get(id=pk) + except Article.DoesNotExist: + raise Exception("Article with id {} does not exist or is not visible".format(pk)) + + context = {'article': article} + + if not request.user.is_authenticated(): + return render(request, + template_name='article.html', + context=context) + + qs = article.payments.filter(user=request.user, purpose='view') + + if qs.count() == 0: + # Generate a new payment + payment, _ = Payment.objects.get_or_create(user=request.user, + article=article, + purpose='view', + satoshi_amount=settings.MIN_VIEW_AMOUNT, + status='pending_invoice') + payment.save() + elif qs.count() == 1: + payment = qs.last() + else: + # This should not happen because there should never be more than one view payment per article per person + raise Exception("Multiple payments detected") + + # User client requests that we check if the payment has been made + if request.GET.get('check'): + print "Checking for payment {}".format(payment.payment_request) + if payment.check_payment(): + print "Payment succeeded!" + else: + print "Payment not received" + + if payment.status == 'pending_invoice': + payment.generate_invoice(request.user, article) + elif payment.status == 'pending_payment': + # Do nothing; display the payment page to user + pass + elif payment.status == 'complete': + # Do nothing; display the article to user + pass + elif payment.status == 'error': + # TODO Optionally implement some kind of error resolution + pass + raise Exception("Payment error") + else: + context['payment_status'] = payment.status + + context['payment'] = payment + + return render(request, + template_name='article.html', + context=context) +``` +{% endraw %} + +A quick breakdown of what's going on: +1. First, we try to find the appropriate `Article` by its id. If successful, we + populate the `context` variable so that the template can work with it. +2. If the user is not logged in, we can't check for a payment from them because + we don't know who they are, so we just direct them to the article page. +3. We attempt to find a `Payment` for this particular `User` and `Article`. If + a `Payment` was not found, we generate a new one. The satoshi amount is set + by `settings.MIN_VIEW_AMOUNT`, which defaults to 1000 satoshis. This newly + created `Payment` has the initial state: `pending_invoice`. +4. If the user provides the `check` query parameter, we call the + `check_payment` function, which will check if the payment was indeed + complete and update the state if so. +5. If the payment is pending an invoice, we call the `generate_invoice` + function, which will generate the invoice and update the Payment state. +6. If the payment is in any other state, we render `article.html` for the user + or throw an error. + +### Adding in a paywall + +Right now, all the articles in the app are freely viewable. We all know no one +likes free stuff, so let's fix that. + +We will begin by adding some conditionals in the html where we are currently +displaying the article content. + +In `coindesk/templates/article.html`, notice the following code: +{% raw %} +```html +
+
+ {{ article.text }} +
+
+``` +{% endraw %} + +{% raw %}`{{ article.text }}`{% endraw %} is the Django template variable +representing the text of the website. We need to hide this for all +non-authenticated users. Let's show them the `pay.html` page instead. And of +course, users who have completed payments must be able to view the article +text. + +Make the necessary edits: +{% raw %} +```html +
+ {% if not request.user.is_authenticated %} +

Make Payment

+

+ To view this article, please log in and return to this page to make + a payment. +

+ Log in + {% elif payment.status == "complete" %} +
+ {{ article.text }} +
+ {% else %} +
+ {% include "pay.html" %} +
+ {% endif %} +
+``` +{% endraw %} + +As this is a Lightning tutorial, understanding the exact syntax is not +important, but a quick clarification may be helpful: +- The {% raw %}`{% if %}`{% endraw %} blocks are Django's way of adding + conditionals in html templates. We are using this to determine if users are + logged in or if the payment is complete. +- `payment` is a context variable the template needs populated in order to + render correctly and access the relevant information. This was done in the + `article` view. +- The {% raw %}`{% include %}`{% endraw %} tag adds in the template by the name + of `pay.html`. + +We should now see something like this when we click on an article: +![logged out article page](http://i.imgur.com/bVc4M06.png) + +### Generating an invoice + +Examine `coindesk/templates/pay.html`: +{% raw %} +```html +
+
+

Make Payment

+

To view this article, please pay 1000 satoshis via the request below:

+

{{ payment.payment_request }}


+ Complete +
+
+``` +{% endraw %} + +This template prompts the user to pay 1000 satoshis, and supplies a payment +request. + +Log in as Alice. The page should look something like this: +![logged in article page with payment prompt missing payment +request](http://i.imgur.com/FpyjezK.png) + +We are missing the payment request, so let's generate an invoice. + +Navigate to `generate_invoice` in `coindesk/models.py`. Modify it to the +following: + +{% raw %} +```python +def generate_invoice(self, user, article): + """ + Generates a new invoice + """ + assert self.status == 'pending_invoice', "Already generated invoice" + channel = grpc.insecure_channel(settings.LND_RPCHOST) + stub = lnrpc.LightningStub(channel) + + add_invoice_resp = stub.AddInvoice(ln.Invoice(amt=settings.MIN_VIEW_AMOUNT, memo="User '{}' | ArticleId {}".format(user.username, article.id))) + r_hash_base64 = codecs.encode(add_invoice_resp.r_hash, 'base64') + self.r_hash = r_hash_base64.decode('utf-8') + self.payment_request = add_invoice_resp.payment_request + self.status = 'pending_payment' + self.save() +``` +{% endraw %} + +Don't forget also to include the necessary imports at the top of the file: +```python +from coindesk import rpc_pb2 as ln, rpc_pb2_grpc as lnrpc +import grpc +``` + +Walking through the code: +- We first check that this `Payment` is indeed pending an invoice, and throw an + error otherwise. +- Using gRPC, we add an invoice of the amount set by `MIN_VIEW_AMOUNT` in + `settings.py`. +- We set the `r_hash` and `payment_request` attributes and update the state of + this `Payment` to now be pending payment. + + +### Checking for payment receipt + +The last piece needed for our fully functional Lightning Coindesk is to +implement the `check_payment` function of the `Payment` class. Recall from +`coindesk.views.` that `check_payment()` is called if the user passes along the +`check` query parameter. For your convenience, we have already implemented this +for you in the frontend: + +In `coindesk/templates/pay.html`: +```html +Complete +``` + +Now, try to implement the `check_payment` function yourself. Here's some +starter code to help you with the annoying and unimportant parts: + +```python +def check_payment(self): + """ + Checks if the Lightning payment has been received for this invoice + """ + if self.status == 'pending_invoice': + return False + + # Prepare r_hash_bytes, which can be passed into the gRPC client + r_hash_base64 = self.r_hash.encode('utf-8') + r_hash_bytes = str(codecs.decode(r_hash_base64, 'base64')) + + # Implement this + payment_settled = False + + if payment_settled: + # Payment complete + self.status = 'complete' + self.save() + return True + else: + # Payment not received + return False +``` + +If you're absolutely stuck, you may refer to the [tutorial +repo](https://github.com/MaxFangX/lightning-coindesk) for the final solution. + +### Conclusion + +Congratulations! You have now built a working Lightning Coindesk app! If you +are looking to take this tutorial to the next level, consider trying one or +more of the following: + +- Integrate error handling in the webapp; see the TODO in + `coindesk.views.article` +- Add P2P payments through the app. Instead of paying the central server, + distribute your micropayment equally between all the people who upvoted + before you. The upvote functionality was intentionally left unimplemented for + this purpose. +- Run this application in a more "production" environment, by using `testnet` + and deploying your webapp online +- Allow the server to accept Litecoin in addition to Bitcoin. + +At this point, you're ready to build with `lnd`. Go forth and bring +Bitcoin to the masses! + +#### Next Steps + +Read through the [LND Overview and Developer Guide](/overview/) to see best +practices and guidelines for implementing LND in your own webapp, or for a +conceptual brushup of the Lightning Network. + +Check out the accessible dev manuals available on our [Guides page](/guides/). + +#### Navigation +- [Return to Stage 3 - RPC Client](/tutorial/03-rpc-client) +- [Return to main tutorial page](/tutorial/) + +### Questions +- Join the #dev-help channel on our [Community + Slack](https://join.slack.com/t/lightningcommunity/shared_invite/enQtMzQ0OTQyNjE5NjU1LWRiMGNmOTZiNzU0MTVmYzc1ZGFkZTUyNzUwOGJjMjYwNWRkNWQzZWE3MTkwZjdjZGE5ZGNiNGVkMzI2MDU4ZTE) +- Join IRC: + [![Irc](https://img.shields.io/badge/chat-on%20freenode-brightgreen.svg)](https://webchat.freenode.net/?channels=lnd) diff --git a/tutorial/index.md b/tutorial/index.md new file mode 100644 index 0000000..24efdd2 --- /dev/null +++ b/tutorial/index.md @@ -0,0 +1,42 @@ +--- +layout: page +title: Tutorial +--- + +### Overview + +This tutorial will get you up to speed with all the skills necessary to work +with [`lnd`](https://github.com/lightningnetwork/lnd). We will be building a +Bitcoin micropayments paywall into a news site! + +Those seeking more conceptual familiarity with the Lightning Network can read +through the [LND Overview and Developer Guide](/overview/). It also contains a +more in-depth explanation of the components covered in this tutorial, and +includes best practices for integrating LND. It is not required knowledge for +completing this tutorial, but we recommend that you at least skim through the +guide before attempting to build your own app. + +The tutorial assumes you have completed installation of Go, `btcd`, and `lnd` +on simnet. If not, the [installation instructions](/guides/installation/) will +guide you through the process. + +### Stages + +Each of the different stages of this tutorial is oriented towards a specific +skill. Beginners should start at [stage 1](/tutorial/01-lncli). Otherwise, jump +wherever you want below! + +- **[Stage 1 - Setting up a local cluster](/tutorial/01-lncli)** + - How to set up a local environment with three nodes that can make payments + with and through one another. + - Gain experience interacting with `lnd` from the command line. +- **[Stage 2 - Web Client](/tutorial/02-web-client)** + - How to run and use a `lnd` web GUI. +- **[Stage 3 - RPC Client](/tutorial/03-rpc-client)** + - How to set up and interact with `lnd` from an RPC client, specifically + `gRPC`. + - Exposure to `lnrpc` documentation + - Set up web development workspace +- **[Stage 4 - Webapp Integration](/tutorial/04-webapp-integration)** + - Adding a micropayments paywall + - Recommended next steps diff --git a/update_and_render.sh b/update_and_render.sh new file mode 100755 index 0000000..ab12a49 --- /dev/null +++ b/update_and_render.sh @@ -0,0 +1,25 @@ +#!/bin/bash + +# Fetch the latest installation docs +curl -o INSTALL.md -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/docs/INSTALL.md +# cp ~/lightning/ln-lnd/src/github.com/lightningnetwork/lnd/docs/INSTALL.md . + +# Fetch the latest Docker guide +curl -o DOCKER-README.md -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/docker/README.md +# cp ~/lightning/ln-lnd/src/github.com/lightningnetwork/lnd/docker/README.md DOCKER-README.md + +# Fetch the latest RPC docs +curl -o python.md -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/docs/grpc/python.md +# cp ~/lightning/ln-lnd/src/github.com/lightningnetwork/lnd/docs/grpc/python.md . + +curl -o javascript.md -s https://raw.githubusercontent.com/lightningnetwork/lnd/master/docs/grpc/javascript.md +# cp ~/lightning/ln-lnd/src/github.com/lightningnetwork/lnd/docs/grpc/javascript.md . + +# Render docs +./render.py + +# Clean up +rm INSTALL.md +rm python.md +rm javascript.md +rm DOCKER-README.md