Choose GitLab for your next open source project

GitLab.com is a competitor of GIthub. It’s a service provider for git-based source code repositories that offers much more than it’s bigger brother. In this post I will try to convince you to try it out for your next project.

GitLab is not only a simple git hosting; its features impact the whole development process, the way of contributing to a project, executing and running tests, protecting source code from changes, more and more.

gitlab-logo-square

1. Private repositories

Many newcomers to the industry struggle to find git hosting for their projects where they learn how to write good code, they experiment and don’t want to share source code, because it is low quality. On Github all you can get is one private repository, on GitLab… they don’t count. Personally I have several private repositories. I host there my projects from studies, some experiments with new languages as well as assignments in LaTeX. I used GitLab with  my friends to share revision notes before finals. I created a new repository, invited them to my projects as administrators  and we could work together on the same project while no one else could see it. GitLab goes a step ahead of this, so that you can create 3 different types of repositories:

Screenshot_20160211_192103
2. Site personalisation

One of its features making GitLab even more user friendly is that you can setup what information you want to see when visiting a project or main page:

Screenshot_20160211_191007

Even better, you can choose which style you want to see source code and commit diffs in! Screenshot_20160211_191136

Do I have to tell you, it’s good for people with vision problems?

3. GitLab CI

GitLab offer continuous integration service (GitLab CI) without runners for most languages. GitLab provides its own CI runner and service that is very easy to integrate into your project. Installation instructions are also very easy to follow. You will find it in project settings. This feature has its advantages and disadvantages compared to Github Travis:

+ You can test your project on multiple platforms. If your code is platform dependent, uses some kernel-API, requires some setup before tests are run, or your customer requires your software to work on Java6, Java7 and Java8, you can just setup 3 runners with 3 different environments and run tests on them. I use it to test my code on older versions of Qt on BSD and newer versions of Qt on Linux.
+ You don’t have to deal with old Ubuntu packages if you relay on newer version of a library. We had this problem in qTox where we were forced to turn off Travis because of outdated sqlcipher library
+ You can use your machines, so you control everything. You can write a script to automatically make a deploy using your known ssh key.

You need to provide your own machine that will run tests for you
You are responsible for security of that machine and code you are running there
It’s not everything. GitLab integrates very nicely with other CIs:

  • Atlassian Bamboo CI
  • Jenkins
  • Drone CI
  • JetBrains TeamCity CI

Probably some more… You can enable integration with CIs and other services under your project Settings -> Services
Screenshot_20160211_193930

No matter which CI integration you choose, you will get the same view in the UI and each pushed commit will trigger CI build. It works a bit differently than naked Jenkins which you have to configure to start builds every x-hours/minutes. GitLab will automatically trigger build on configured CI when ^HEAD of a branch changes. That means, you can make 10 local commits and push to remote once, so that code will be build once in the CI, or you can push each commit to build it 10 times. Functionality covers also ignoring CI. When you make change in documentation or formatting, you don’t want to waste time, so you can add ‘[skip ci]’ text at the end of the commit message!
Screenshot_20160211_194644

It integrates that nicely, that you can forget that you have Jenkins:
Screenshot_20160211_194857

Or maybe you’re interested in success rate of your builds? Sure, you can see it on a graph.

4. Supporting open source culture

A couple of years ago SourceForge was one of the biggest supporters of OSS. You could host there your source code, huge binary files like ISO images. The problem with SF was that… it was proprietary itself and a lot of people didn’t and still don’t trust it. When git was created, there was a market gap for more secure, more open source friendly hosting. Github was created to fill the gap, but again… they claim to be open source friendly and at the same time they do not publish their code. Github is too expensive for self hosted solutions. They concentrate probably most on open source projects right now. I know business is business, but let’s take a look at how GitLab does it.

GitLab offers community edition source code on MIT license; they host enterprise edition for you for free on gitlab.com. GitLab encourages project managers to specify license, publish and update changelog files and contribution guide. You’re not forced to do it, but it’s a really good practice in open source communities do to it. All of those you can see from the main page of a project:
Screenshot_20160211_200214

If you want to, you can host GitLab CE on your server for free. It’s much easier than you think and it runs everything from Raspberry PI to clusters with 30 000 users. Find out more here.

5. Merge requests approves and functionality

When your project becomes more popular, it requires more and more people to look after it. You need to be careful about changes in source code and its quality. You need more eyes on the source code… or you can just automate it. In project settings you can select a list of trusted people who can approve merge requests and number of required approvals to merge changes. It’s enough to get a few of trusted developers who care about your project, they can vote if diff is good quality and can be merged to main code base.

Screenshot_20160211_201510

It offers another good function, if you reviewed changes, but CI is still running, you can enable ‘auto-merge’. Merge request will be merged after CI reports green!merge_on_pass
6. Issues

Yes, GitLab has issues. Quite serious. It’s obvious, issue can belong to milestone, can have a few labels and be assigned to a user. On GitLab an issue can also have:

  • weight – how important for developers a change is
  • votes – users can vote on issues important for them! This is what I call contribution to open source communities!
  • labels – I didn’t find out how this can be useful… but at least it’s not disturbing. You can give yourself a medal for reporting an issue… I guess

Screenshot_20160211_202620

Now when someone opens me a new issue, they see content of the template in text area 🙂 Isn’t that great? How much time would it save you if you had it 10 years ago? Why don’t you have it yet?

7. Templates!

How often someone opens new issue for a topic that was discussed many times, or users didn’t bother to search for duplicates, or they didn’t try to walk around an issue, or “DOESNT WORK”, or didn’t provide enough details… or… you what. GitLab let’s you specify template of issue and merge request. You can at least try forcing users to deliver you information you need before they click submit. It makes your life easier, saves you a lot of time, helps producing better quality reports and you don’t have to ask the same questions over and over again to every reporter.
Screenshot_20160211_203559

8. Milestones are interesting things in GitLab

They work a bit like sprint boards… You can see open, ongoing and completed issues and progress of the whole milestone. It’s a bit like poor man’s JIRA.
Screenshot_20160211_204115
We’re getting closer to the end of the list of things I like in GitLab, but remember this list is not complete and you will find more features like protected branches, handy way of creating git hooks to prevent committing .exe, .id, .bin, .o files to your repository, limiting max. file size, simple code review editor for diffs.

GitLab also promotes their fork of git-flow called GitLab flow that similarly as GitHub-flow fixes some issues with bare git-flow, take a read. Some other features that you might want to use in your company are LDAP support, repository mirroring.

9. Groups and permissions

As I mentioned in 1. you can create a group. A group is a list of people who share repositories which can be public, internal or private. Groups are always public, but members of groups have permissions. This systems is more complex, so I will refer you to the official documentation where it’s explained better than I could have explained it.

10. Do you value linear git history?

They solved it on merge request level. You can decide what method will be used to add new commits to your branch. If you don’t mind non-linear history, default settings are good; if you want to avoid merge commits, you can force contributors to rebase their commits on ^HEAD of your branch before merging. This way you can also improve code quality, even if there is no git conflict, there might be a change in source code that breaks tests. Learn more about linear history here.
Screenshot_20160211_205653

11. Puzzles aren’t complete

There are some things that GitLab is missing. Many people say it’s web-based notifications. Currently only email-based is supported and more feature complete Android client. Can you live without that?


Original URL: http://feedproxy.google.com/~r/feedsapi/BwPx/~3/M4RgPMEvXyo/

Original article

SourceForge Acquisition and Future Plans

At the end of January, SourceForge and Slashdot were sold to BIZX, LLC by DHI Group, Inc. As the new owners of two iconic sites, we are excited about the future and what we can do together. We’ve already started to take action, and are developing further plans for the site. We encourage your feedback to help us shape the future direction for the site.

Our first order of business was to terminate the “DevShare” program. As of last week, the DevShare program was completely eliminated. The DevShare program delivered installer bundles as part of the download for participating projects. We want to restore our reputation as a trusted home for open source software, and this was a clear first step towards that. We’re more interested in doing the right thing than making extra short-term profit. As we move forward, we will be focusing on the needs of our developers and visitors by building out site features and establishing community trust. Eliminating the DevShare program was just the first step of many more to come. Plans for the near future include full https support for both SourceForge and Slashdot, and a lot more changes we think developers and end-users will embrace.

Stay tuned for future announcements about how we’re making SourceForge better for everyone.

Logan Abbott

President

SourceForge Media, LLC


Original URL: http://feedproxy.google.com/~r/feedsapi/BwPx/~3/z9cfFDs8hvc/

Original article

LinkedIn Is Open Sourcing Their Testing Frameworks

destinyland writes: LinkedIn is open sourcing their testing frameworks, and sharing details of their revamped development process after their latest app required a year and over 250 engineers. Their new paradigm? “Release three times per day, with no more than three hours between when code is committed and when that code is available to members,” according to a senior engineer on LinkedIn’s blog. This requires a three-hour pipeline where everything is automated, from committing code to releasing it into production, along with automated analyses and testing. “Holding ourselves to this constraint ensures we won’t revert to using manual validation to certify our releases.”


Share on Google+

Read more of this story at Slashdot.


Original URL: http://rss.slashdot.org/~r/Slashdot/slashdot/~3/_NHKAOZJjq0/linkedin-is-open-sourcing-their-testing-frameworks

Original article

Unlocking my Lenovo laptop, part 3

The decryption function

If you are just joining this story you may want to start at part 1.

In part 2, we discovered that a embedded controller update is performed by uploading a small ‘flasher’ program to the EC. This flasher program is then responsible for programming a new firmware image to the EC’s internal flash memory. However, both the flasher program and part of the firmware image are encrypted: the old (currently running) EC firmware decrypts the flasher program, and the flasher program then decrypts the new firmware update. This creates a bit of a chicken-and-egg problem that prevents discovering the encryption algorithm from firmware update files alone.

We managed, however, to find a decrypted version of the EC firmware online, dumped directly from EC flash memory via JTAG. Let’s dive right in and disassemble the decryption function that can be found in that flash image. The core of it looks like this:

2854:  30 25 8d 1f 00 00 48 14      ld         r13,[r13,0x1448]
285c:  30 21 81 0f 00 00 48 10      ld         r1,[r1,0x1048]
2864:  cf 7e                        extb_s     r14,r14
2866:  02 be                        asl_s      r14,r14,2
2868:  b9 61                        add_s      r1,r1,r13
286a:  30 26 8d 1f 00 00 48 18      ld         r13,[r14,0x1848]
2872:  4f 7f                        extb_s     r15,r2
2874:  02 bf                        asl_s      r15,r15,2
2876:  a7 79                        xor_s      r1,r1,r13
2878:  30 27 8d 1f 00 00 48 1c      ld         r13,[r15,0x1c48]
2880:  b9 61                        add_s      r1,r1,r13

Here each input byte is transformed through a lookup table (in cryptography terminology, a substitution box or ‘S-box’) and the results are combined with an add/xor/add structure. This is the Blowfish cipher, as becomes evident from one glance at the diagram in the Wikipedia article on Blowfish:

Blowfish F function

[Diagram by Decrypt3/DnetSvg via Wikimedia, CC BY-SA 3.0]

Now normally the first stage of Blowfish, like most ciphers, would be to expand a user-provided key – a password or passphrase or some other secret data – to produce a set of 18 round keys (called the ‘P array’ in Blowfish terminology) and the four 256-entry S boxes depicted above. In cryptography this expansion step is called a key schedule.

In the case of the Lenovo firmware, it turns out that the keys are stored in pre-expanded form, i.e. the values of the P array and S boxes are stored in flash memory rather than the original key string. We can extract the P array and S boxes from the dumped flash image and use them for encryption/decryption.

(I do not believe there is any easy way – where in cryptography easy means ‘significantly better than trying all possibilities’ – to recover the original key string that was used to generate the P array and S boxes. This would be an interesting challenge, but is of purely academic interest: the P array and S boxes are all that is needed for both encryption and decryption. Each round of Blowfish involves taking the data, mixing in [exclusive or] one of the round keys from P and then performing the function depicted above using the S boxes; this is repeated 16 times; apart from some minor details you now understand how Blowfish works.)

The checksum function

Having the encryption algorithm and keys, we can now decrypt the flasher program that is uploaded to the embedded controller when performing a firmware update.

Analysis of the flasher program allows us to determine the checksum algorithm used to validate the firmware image. Even without a detailed analysis of the disassembly, we can notice that it uses the following table:

uint16_t csum_table[256] = {
 0x0000, 0x1021, 0x2042, 0x3063, 0x4084, 0x50A5, 0x60C6, 0x70E7,
 0x8108, 0x9129, 0xA14A, 0xB16B, 0xC18C, 0xD1AD, 0xE1CE, 0xF1EF,
 0x1231, 0x0210, 0x3273, 0x2252, 0x52B5, 0x4294, 0x72F7, 0x62D6,
 0x9339, 0x8318, 0xB37B, 0xA35A, 0xD3BD, 0xC39C, 0xF3FF, 0xE3DE,
 ...
};

A quick Google for these numbers gives this secret away easily: this is a lookup table for a CRC-16 function with a generator polynomial of 0×1021. At this point we can be pretty sure that the algorithm is CRC-16, with only a few details to determine like initialisation value, but let me indulge myself and make a brief digression into what a CRC is and why the table looks like this. You can skip to the next section when this gets too dense for your liking.

The first thing to note is that when people refer to a CRC-16 generator polynomial of 0×1021, what they actually mean is 0×11021, i.e. binary 10001000000100001, i.e. x^16 + x^12 + x^5 + 1. (The x^n term of a CRC-n generator polynomial is always assumed to be 1, otherwise the following algorithm would not produce an n-bit CRC.)

Now the best way to imagine calculating a CRC is, for every 1 bit that is set in the input message, you XOR in the polynomial (towards the right, if working left to right). This clears that bit and flips some subsequent bits: for the above polynomial, it flips the three bits at N+4,N+11,N+16. If those bits end up as 1 (when you come to them), they will in turn propagate to other bits. Once you reach the end of the message, you’ve cleared all the bits in the message, but you have some bits hanging off the end: these are the CRC. You can intuitively see from this that the CRC has error propagating properties; an erroneous bit will likely cascade to many other bits. (Engineers might be reminded of a linear feedback shift register which has many similarities. The CRC can also be expressed mathematically as the remainder of a long division in carry-less binary arithmetic [GF(2)], the similarity to long division will become apparent in the examples below.)

As an example, if the input message to our CRC is one byte with value 5 (00000101), then the CRC calculation would proceed like this, where I have highlighted in red the leading 1 at each step:

        (5)  00000101|
              xor 100|01000000100001 (0x11021)
             -----------------------
             00000001|01000000100001
                xor 1|0001000000100001 (0x11021)
             -------------------------
             00000000|0101000010100101 (0x50a5)

Here is another example for an input byte with value 16 (00010000):

       (16)  00010000|
            xor 10001|000000100001 (0x11021)
             ---------------------
             00000001|000000100001
                xor 1|0001000000100001 (0x11021)
             -------------------------
             00000000|0001001000110001 (0x1231)

For larger messages doing this bit by bit would be slow, of course. To do it more efficiently we can pre-calculate the effect of a byte on the next sixteen bits (this will be some pattern of bit flips), for each of the 256 possible values of that byte. We then look up the first byte, XOR that in to the next sixteen bits, move to the next byte and repeat. Actually the “effect of the first byte on the next sixteen bits” is exactly equivalent to the CRC of that byte – the CRC is the pattern of bit flips that ends up past the end of a message – so the lookup table is in effect a table of CRCs of single bytes.

Looking at the above examples you can verify that the CRC of 5 is indeed the entry with index 5 in the table above, and the CRC of 16 is indeed the entry with index 16. If you stare at the first example for a moment, you might see why the first sixteen entries of the table are multiples of 0×1021: all the 1 bits in 0×11021 are far enough apart that the input bits propagate independently into the CRC. Things change at entry 16 because the next 1 in the polynomial crosses over into the message.

Now, back to your regularly scheduled programming

Returning to the firmware, we can verify that the flasher checksum is in fact CRC-16. It’s important to note that the flasher program calculates this checksum after decrypting the firmware image, so the CRC-16 must be calculated on the decrypted version of the image. (It turns out that the firmware image is encrypted using the same key as the flasher program, so at this point we already have everything we need to know.)

I mentioned in part 2 that a simple 16-bit checksum is also present at the very end of the firmware image. In fact, there is also a third type of checksum that we discover when we disassemble the EC boot code: four 32-bit checksums, each performed on a different area of the flash memory.

Summarising now, the three EC checksums that must be correct are:

  • The outer checksum: BIOS checks the checksum of the (encrypted) image before uploading it to the flasher. The last two bytes of the image are calculated such that the total sum of the 16-bit words of the image is zero. If this fails, flashing doesn’t proceed, so this is the least dangerous checksum to fail.
  • The flasher checksum: The flasher calculates the CRC-16 checksum of all the decrypted bytes except the last four bytes. The penultimate two bytes of the image contain this checksum. If this checksum fails, the flasher sets a flag in flash that causes the EC firmware to fail to boot.
  • The boot checksums: The EC firmware, at boot, calculates 32-bit checksums of four different areas of the flash. If any of these fail the EC firmware fails to boot. These checksums must, of course, also be calculated on the decrypted image.

Building a modified firmware

We now have everything we need to know to successfully modify the embedded controller firmware. Returning to my original goal, here is the change I made to the battery authentication check (before/after, with some comments added):

  ; call validation function on battery response
  1d168:   ee 0f af f2         bl.d       0x2954 ; validate
  1d16c:   f8 16 02 11         ldw        r2,[r14,248]
- ; branch to failure path if return value not equal to 1
- 1d170:   0b 08 51 00         brne       r0,1,0x1d17a
+ ; branch replaced with no-operation instruction
+ 1d170:   4a 26 00 70         nop        

  ; success path - set bit 3 in battery status
  1d174:   00 86               ld_s       r0,[r14,0]
  1d176:   83 b8               bset_s     r0,r0,3
  1d178:   24 f0               b_s        0x1d1c0

  ; failure path - try a different validation function,
  ;                else retry up to 8 times, else abort
  1d17a:   10 10 80 20         ldb        r0,[r16,16]
  ...

Previously, if the return value from the validation function was not equal to 1, the code would branch to a failure path. I replaced this conditional branch instruction with a no-operation instruction so that, regardless of the validation result, execution would fall through to the success path. (This technique of replacing jumps with nops is a common trick when you need to modify the behaviour of binary code.)

(At this point my focus was on the first authentication sequence – state 12 in the firmware state machine that I listed in part 2.)

Also, for fun and so I could track my modified firmware version, I changed the embedded firmware version from “GCHT25WW” to “GCHT25MC”. I thought putting my initials in the geographic region field would be appropriate: the original suffix, WW, presumably means worldwide, while this firmware had… somewhat more limited geographic reach.

Having made my changes, I wrote some small utilities to regenerate the three types of checksums – I will publish these utilities shortly – and finally I re-inserted the modified embedded controller firmware into the BIOS update .FL2 file at offset 0×500000 (dd if=ec.bin of=$01DA000.FL2 conv=notrunc bs=4096 seek=1280).

If you are attempting this, make sure you check and double-check your work here: it’s worth re-extracting the embedded controller image from the BIOS update file, re-verifying the outer checksum, re-decrypting the image and re-verifying the inner checksums, and making sure that the changes are exactly as intended (for Linux users, process substitution can be handy here, e.g. to get a binary diff of two files you can do: diff -u

The moment of truth?

I ran the BIOS update program and was greeted with:

An update is not necessary at this time. The process has been canceled.

Thwarted. I was already running the latest BIOS version and the update program did not offer an option to force an update of the BIOS or EC. Downgrading and upgrading might usually be a workaround, but I wasn’t sure that this would be possible as the Lenovo release notes mention that, “if the UEFI BIOS has been updated to version 2.63 or higher, it is no longer able to roll back to the version before 2.63 for security improvement”.

Fortunately, it turns out it’s possible to run the update program manually, bypassing the limited user interface.

Manually updating the EC

A note: As I do not have Windows on this laptop, I am running the BIOS update program from a USB flash drive following some instructions I found online (in short: download the bootable CD image; run geteltorito -o bios.img gcuj23us.iso; write the bios.img file to the USB stick). I suspect the below process is even easier from Windows, where you can directly run winflash32.exe or winflash64.exe in place of dosflash.exe.

The Lenovo BIOS update CD is just a DOS boot disk at heart. If you’ve written it to rewritable media like a USB flash drive, you can edit autoexec.bat to stop it starting the user interface, in which case it will drop you to a DOS prompt.

Updating the EC firmware can be performed using dosflash.exe as follows (/sd is skip date check to allow downgrading, /ipf ec targets the EC area):

C:FLASH> dosflash /sd /ipf ec /file GCETA3WW$01DA000.FL2
SCT Flash Utility for Lenovo
 for Shell V1.0.1.3
Copyright (c) 2011-2012 Phoenix Technologies Ltd.
Copyright (C) 2011-2012 Lenovo Group Limited.

Read BIOS image from file.
Initialize Flash module.
Read current BIOS.
Oem check

Prepare to flash “ec”

Do not turn off computer during the update!!!

Begin Flashing……
Total blocks of the image = 48.
|—+—-+—-+—-+—-+—-+—-+—-+—-+—-|
…………………………………………..
Image flashing done.

Flashing finished.

BIOS is updated successfully.

WARNING: System will shutdown or reboot in 5 seconds!

A note here on the FL1 and FL2 files since I couldn’t find any explanation about this on the Internet: the FL1 file is a UEFI capsule file that contains the BIOS. The FL2 file contains embedded controller firmware at offset 0×500000-0×530000, the rest of it can be ignored. Why then is the FL2 file so large and why does it contain bits of an old BIOS version pasted after the EC firmware? I think partly it may be to appease the non-Lenovo-specific parts of dosflash.exe. I noticed that even though ultimately it only uses the 48 4KB blocks from 0×500000-0×530000, if I pass a file that ends there, dosflash.exe does not recognise it as a valid BIOS update file.

(While the command shown above updates the EC, I will note here that it is also possible to update the BIOS in a similar way, by omitting /ipf ec and by specifying the FL1 file instead of the FL2 file: dosflash /sd /file GCETA3WW$01DA000.FL1
Of course I recommend using the the normal manufacturer-recommanded BIOS upgrade/downgrade process when possible, but this may be useful if you are in a bind.)

Note that, despite what the output of dosflash.exe says, the actual EC firmware is not updated yet: at this point it has just written the update to an area where it can be picked up by BIOS at boot. Now after reboot the screen displays:

Flashing Embedded Controller…
Please do not power off!

(I appreciate the ‘please’, but really that should read “DO NOT UNDER ANY CIRCUMSTANCES POWER OFF, EVEN IF YOUR HOUSE IS ON FIRE AND SOMEONE HAS STOLEN YOUR PANTS”.)

A few skipped heartbeats later, the firmware update completes.

The moment of truth?

ec

Sure enough, the system is now running my modified embedded controller firmware.

But my battery still isn’t charging. I hook up the SMBus signals to my logic analyser again and the communication looks like this:

...
START 16 (Control Byte: Slave Address B Write) 3C 04 NACK STOP 
START 16 (Control Byte: Slave Address B Write) 3C 04 NACK STOP 
START 16 (Control Byte: Slave Address B Write) 3C 04 NACK STOP 
START 16 (Control Byte: Slave Address B Write) 3C 04 NACK STOP 

It turns out that it isn’t even getting to the authentication check that I had modified, because the earlier command that sends the challenge to the battery is failing: as soon as the laptop sends command 3C and the data length indication 04, the battery is signalling NACK – not acknowledged – go away. So now I modify the state machine so that it proceeds whether or not that write command succeeds (again I do this by replacing a jump by a nop, this time in state 8).

Revision two deployed. Now the system boots with:

The battery installed is not supported by this system and will not charge. Please replace the battery with the correct Lenovo battery for this system. Press the ESC key to continue.

Well, that’s some sort of progress: at least it is no longer displaying the original unauthorised battery message.

I look in BIOS to see where these messages are coming from. Both this message and the original unauthorised battery message are displayed by LenovoVideoInitDxe.efi: don’t ask me why this code is in this module rather than somewhere more relevant (may I suggest LenovoAnnoyingBatteryMessageDxe.efi?), but it might have been convenient to put it in the video initialisation module as the message is displayed when the screen is cleared post-POST.

Anyway, in LenovoVideoInitDxe.efi it reads the battery status from the EC (register 0×38, which we came across in part 2 when decoding the ACPI tables, as well as register 0xd1 which has some additional battery status flags). Depending on certain bits in those registers, it may print one or other message.

Tracing through the EC code to find where those bits are set proves difficult, so I again hook the battery up to my logic analyser. It becomes evident that the sticking point is now the second authentication sequence.

I now make the same changes to the second authentication sequence as I did for the first: I prevent the write command from failing even if the battery NACKs it (state 15), and force the check of the response to succeed (state 19). This is now four changes in total, for each of which I’ve replaced a jump with a nop.

After booting to this final EC firmware revision, my saga comes to an end, almost anticlimactically. My replacement battery works, and I’m getting a good few hours out of it (and no, it hasn’t burst into flames).

Postscript

There is still one very curious open question that I haven’t managed to figure out. There are 10 random-looking bytes at offset 0×200 of the firmware image – the start of what looks like a firmware information block – which are different in every EC revision. So far I haven’t found anything that accesses those bytes, and indeed my EC update works fine even when I leave them unchanged. Probably this is a red herring, but what these bytes are for is still a mystery.

In the next few days I will upload the utilities I wrote so that it is possible to replicate my work and possibly make other changes to EC firmware, however a large disclaimer applies: do not attempt EC firmware modification at home unless you understand what you are doing. If something goes wrong with an EC update, there is a high likelihood of bricking your laptop, the only recourse being connecting to the EC via JTAG. I will not be held responsible for this. You should also understand that poor quality lithium ion cells can cause fires, as has been seen in the recent spate of hoverboard fires. I will also not be held responsible for this.

I have since also torn down my old Lenovo battery, and I plan to write another post soon with some information and photos. The option of replacing the cells in a genuine battery may be worth considering as an alternative to modifying the EC firmware, the advantage being is that you can choose your own high quality Li-Ion cells versus whatever you might happen to get in a replacement battery. The disadvantage is that it inevitably results in some damage to the battery casing, and as I mentioned before the controller will remember data about the old cells which might affect function of the new cells (I will see what I can figure out on that front).

To be fair, buying a genuine Lenovo battery is probably the best option for most people, at least while Lenovo is still making replacement batteries for this model. Primarily this was an exercise in ‘because I can’: I and a great many readers have enjoyed this process of diving deep into the system architecture of a modern laptop, at a level that few people are normally exposed to, and removing an annoying limitation that I feel I should be entitled to remove from my laptop. I do not have anything against Lenovo specifically – they are certainly not the only vendor who implements vendor restrictions – so please be nice in your comments.


Original URL: http://feedproxy.google.com/~r/feedsapi/BwPx/~3/qRqkO5i_qyY/

Original article

Awesome Web Development Tools and Resources

The best and worst thing about being a web developer is that the web is constantly changing. While this is exciting it also means that web developers must always be proactive about learning new techniques or programming languages, adapting to changes, and be willing and eager to accept new challenges. This could include tasks such as adapting existing frameworks to meet business requirements, testing a website to identify technical problems, or optimizing and scaling a site to better perform with the back-end infrastructure.  We thought we would compile a comprehensive list of web development tools and resources that can help you be more productive, stay informed, and become a better developer.

Web Development Tools and Resources

A lot fo these web development tools below are ones we use at KeyCDN on a daily basis. We can’t include everything, but here are a couple of our favorites and other widely used ones. Hopefully, you find a new tool or resource that will aid you in your development workflow. Note: The tools and resources below are listed in no particular order.

JavaScript Libraries

javascript libraries

Javascript is one of the most popular programming languages on the web. A Javascript library is a library of pre-written Javascript which allows easier access throughout the development of your website or application. For example, you can include a copy of Google’s hosted jQuery library by using the following snippet.


  • jQuery: A fast, small, and feature-rich JavaScript library.
  • BackBoneJS: Give your JS app some backbone with models, views, collections, & events.
  • D3.js: A JavaScript library for manipulating documents based on data.
  • React: Facebook’s Javascript library developed for building user interfaces.
  • jQuery UI: A curated set of user interface interactions, effects, widgets, and themes.
  • jQuery Mobile: HTML5-based user interface system designed to make responsive web sites.
  • Underscore.js: Functional programming helpers without extending any built-in objects.
  • Moment.js: Parse, validate, manipulate, and display dates in JavaScript.
  • Lodash: A modern utility library delivering modularity, performance, & extras.

Front-end Frameworks

web development tools - front-end frameworks

Front-end frameworks usually consist of a package that is made up of other files and folders, such as HTML, CSS, JavasScript, etc. There are also many stand-alone frameworks out there. We are a big fan of Boostrap and the main KeyCDN website is built on it. A solid framework can be an essential tool for front-end developers.

  • Bootstrap: HTML, CSS, and JS framework for developing responsive, mobile first projects on the web.
  • Foundation: Family of responsive front-end frameworks that make it easy to design beautiful responsive websites, apps and emails that look amazing on any device.
  • Semantic UI: Development framework that helps create beautiful, responsive layouts using human-friendly HTML.
  • uikit: A lightweight and modular front-end framework for developing fast and powerful web interfaces.

Web Application Frameworks

web application frameworks

A web application framework is a software framework designed to aid and alleviate some of the headache involved in the development of web applications and services. KeyCDN actually hosts a version of AngularJS, HTTP/2 and CORS enabled, that you can use in your projects completely free. See code below. Full documentation at angularcdn.com.

https://opensource.keycdn.com/angularjs/1.4.8/angular.min.js
  • Ruby: Ruby on Rails is a web-application framework that includes everything needed to create database-backed web applications, with the MVC pattern.
  • AngularJS: Lets you extend HTML vocabulary for your web application. AngularJS is a framework, even though it’s much more lightweight and sometime referred to as a library.
  • Ember.js: A framework for creating ambitious web applications.
  • Express: Fast and minimalist web framework for Node.js.
  • Meteor: Full-stack JavaScript app platform that assembles all the pieces you need to build modern web and mobile apps, with a single JavaScript codebase.
  • Django: High-level Python Web framework that encourages rapid development and clean, pragmatic design.
  • ASP.net: Free, fully supported Web application framework that helps you create standards-based Web solutions.
  • Laravel: A free, open-source PHP web application framework to build web applications on MVC pattern.
  • Zend Framework 2: An open source framework for developing web applications and services using PHP.
  • Phalcon: A full-stack PHP framework delivered as a C-extension.
  • Symfony: A set of reusable PHP components and a web application framework.
  • CakePHP: A popular PHP framework that makes building web applications simpler, faster and require less code.
  • Flask: A microframework for Python based on Werkzeug and Jinja 2.
  • CodeIgniter: Powerful and lightweight PHP framework built for developers who need a simple and elegant toolkit to create full-featured web applications.

Also make sure to check out KeyCDN’s framework integration guides to see how you can implement a CDN with the solutions mentioned above.

Task Runners / Package Managers

build systems task runners

Tasks runners are all about automating your workflow. For example you can create a task and automate the minification of JavaScript. Then build and combine tasks to speed up development time. Package managers keep track of all the packages you use and make sure they are up to date and the specific version that you need.

  • Grunt: JavaScript task runner all about automation.
  • Gulp: Keeps things simple and makes complex tasks manageable, while automating and enhancing your workflow.
  • npm: Pack manager for JavaScript.
  • Bower: A web package manager. Manage components that contain HTML, CSS, JavaScript, fonts or even image files.

Languages / Platforms

programming languages

Behind all the web development tools is a language. A programming language is a formal constructed language designed to communicate with a computer and create programs in which you can control the behavior. And yes we realize some of these might not always be referred to as a language.

  • PHP: Popular general-purpose scripting language that is especially suited to web development.
  • NodeJS: Event-driven I/O server-side JavaScript environment based on V8.
  • Javascript: Programming language of HTML and the web.
  • HTML5: Markup language, the latest version of HTML and XHTML.
  • Python: Programming language that lets you work quickly and integrate systems more effectively.
  • Ruby: A dynamic, open source programming language with a focus on simplicity and productivity.
  • Scala: Scala is a pure-bred object-oriented language allowing a gradual, easy migration to a more functional style.
  • CSS3: Latest version of cascading style sheets used in front-end development of sites and applications.
  • SQL: Stands for structured query language used with relational databases.
  • Golang: Open source programming language that makes it easy to build simple, reliable, and efficient software.
  • Rust: Systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.
  • Elixir: Dynamic, functional language designed for building scalable and maintainable applications.

Databases

databases

A database is a collection of information that is stored so that it can be retrieved, managed and updated.

  • MySQL: One of the world’s most popular open source databases.
  • MariaDB: Made by the original developers of MySQL. MariaDB is also becoming very popular as an open source database server.
  • MongoDB: Next-generation database that lets you create applications never before possible.
  • Redis: An open source, in-memory data structure store, used as database, cache and message broker.

CSS Preprocessors

web development tools - css preprocessors

A CSS preprocessor is basically a scripting language that extends CSS and then compiles it into regular CSS. Make sure to also check out or in-depth post on Sass vs Less.

  • Sass: A very mature, stable, and powerful professional grade CSS extension.
  • Less: As an extension to CSS that is also backwards compatible with CSS. This makes learning Less a breeze, and if in doubt, lets you fall back to vanilla CSS.
  • Stylus: A new language, providing an efficient, dynamic, and expressive way to generate CSS. Supporting both an indented syntax and regular CSS style.

If you are just getting started with a CSS preprocessor you might want to make the transition easier by first using a 3rd party compiler, such as the ones below.

Text Editors / Code Editors

text editors code editors

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Pellentesque fringilla, lorem sit amet aliquet porttitor, sapien nunc aliquam risus, et porta erat nibh vitae leo. Pellentesque non faucibus nibh. Fusce sollicitudin ac sapien nec semper.

  • Atom: A text editor that’s modern, approachable, yet hackable to the core. One of our favorites!
  • Sublime Text: A sophisticated text editor for code, markup and prose with great performance.
  • Notepad++: A free source code editor which supports several programming languages running under the MS Windows environment.
  • Visual Studio Code Beta: Code editing redefined and optimized for building and debugging modern web and cloud applications.
  • TextMate: A code and markup editor for OS X.
  • Coda 2: A fast, clean, and powerful text editor for OS X.
  • WebStorm: Lightweight yet powerful IDE, perfectly equipped for complex client-side development and server-side development with Node.js.

Markdown Editors

markdown editors

Markdown is a markup language in plain text using an easy syntax that can then be converted to HTML on the fly. Note: This is different than a WYSIWYG editor. Markdown editors are sometimes referred to as the in-between WYSIWYG and simply writing code.

  • StackEdit: A free online rich markdown editor.
  • Dillinger: An online cloud-enabled, HTML5, buzzword-filled Markdown editor.
  • Mou: Markdown editor for developers on Mac OS X.
  • Texts: A rich editor for plain text. Windows and Mac OS X.

Some of the text editors we mentioned above also support markdown. For example, there is a markdown preview package for atom.

Icons

icons

Almost every web developer, especially front-end developers will at some point or another need icons for their project. KeyCDN actually hosts a version of Font Awesome, HTTP/2 and CORS enabled, that you can use in your projects completely free. See code below. Full documentation at fontawesomecdn.com.

https://opensource.keycdn.com/fontawesome/4.5.0/font-awesome.min.css
  • Font Awesome: Scalable vector icons that can instantly be customized — size, color, drop shadow, and anything that can be done with the power of CSS.
  • IconMonster: A free, high quality, monstrously big and continuously growing source of simple icons. One of our favorites!
  • IconFinder: Iconfinder provides beautiful icons to millions of designers and developers.
  • Fontello: Tool to build custom fonts with icons.

Git Clients / Services

git clients

Git is a source code management system for software and web development known for distributed revision control. When working with teams, using a git client to push code changes from dev to production is a way to maintain the chaos and ensure things are tested so they don’t break your live web application or site.

  • SourceTree: A free Git & Mercurial client for Windows or Mac. Atlassian also makes a pretty cool team Git client called Bitbucket.
  • GitKraken (Beta):  A free, intuitive, fast, and beautiful cross-platform Git client.
  • Tower 2: Version control with Git – made easy. In a beautiful, efficient, and powerful app.
  • GitHub Client: A seamless way to contribute to projects on GitHub and GitHub Enterprise.
  • Gogs: A painless self-hosted Git service based on the Go language.

Local Dev Environments

local dev environment

Depending upon what OS you are running or the computer you currently have access to, it might be necessary to launch a quick local dev environment. There are a lot of free utilities that bundle Apache, mySQL, phpmyAdmin, etc. all together. This can be a quick way to test something on your local machine. A lot of them even have portable versions.

  • XAMPP: Completely free, easy to install Apache distribution containing MariaDB, PHP, and Perl.
  • MAMP: Local server environment in a matter of seconds on OS X or Windows.
  • WampServer: Windows web development environment. It allows you to create web applications with Apache2, PHP and a MySQL database.

Diff Checkers

diff checkers

Diff checkers can help you compare differences between files and then merge the changes. A lot of this can be done from CLI, but sometimes it can be helpful to see a more visual representation.

  • Diffchecker: Online diff tool to compare text differences between two text files. Great if you are on the go and quickly need to compare something.
  • Beyond Compare: A program to compare files and folders using simple, powerful commands that focus on the differences you’re interested in and ignore those you’re not.

A lot of the free text editors we mentioned above also have plugins or extensions which allow you to diff or compare your files.

Code Sharing / Experimenting

code sharing

There is always that time when you are on Skype or Google hangout with another developer and you want him or her to take a quick look at your code. There are great team tools for sharing code like Slack, but if they aren’t a member of your team there are some great quick alternatives. Remember not to share anything secure.

  • JS Bin:  Tool for experimenting with web languages. In particular HTML, CSS and JavaScript, Markdown, Jade and Sass.
  • JSfiddle: Custom environment to test your JavaScript, HTML, and CSS code right inside your browser.
  • codeshare: Share code in real-time with other developers.
  • Dabblet: Interactive playground for quickly testing snippets of CSS and HTML code.

Collaboration Tools

dev collaboration tools

Every great development team needs a way to stay in touch, collaborate, and be productive. A lot of teams work remotely now. The team at KeyCDN is actually spread across many different continents. Tools like these below can help employees streamline their development workflow.

  • Slack: Messaging app for teams that is on a mission to make your working life simpler, more pleasant, and more productive. One of our favorites, we use this at KeyCDN!
  • Trello:  Flexible and visual way to organize anything with anyone. We also use this as KeyCDN.
  • Glip: Real-time messaging with integrated task management, video conferencing, shared calendars and more.
  • Asana: Team collaboration tool for teams to track their work and results.
  • Jira: Built for every member of your software team to plan, track, and release great software or web applications.

Inspiration

developers inspiration

We all need inspiration at some point or another. For front-end developers especially, from time to time, it can be beneficial to check out what other people are doing. This can be a great source of inspiration, new ideas, and making sure your web application or site doesn’t fall behind the times.

  • CodePen: Show off your latest creation and get feedback. Build a test case for that pesky bug. Find example design patterns and inspiration for your projects.
  • Dribble: A community of designers sharing screenshots of their work, process, and projects.

Website Speed Test Tools

website speed test tools

The speed of a website can be a critical factor to its success. Faster loading websites can benefit from higher SEO rankings, higher conversion rates, lower bounce rates, and a better overall user experience and engagement. It is important to take advantage of the many free tools available for testing website speed.

  • Website Speed Test:  A page speed test developed by KeyCDN that includes a waterfall breakdown and the website preview.
  • Google PageSpeed Insights: PageSpeed Insights analyzes the content of a web page, then generates suggestions to make that page faster.
  • Google Chrome DevTools: Set of web authoring and debugging tools built into Google Chrome.
  • WebPageTest: Run a free website speed test from multiple locations around the globe using real browsers (IE and Chrome) and at real consumer connection speeds.
  • Pingdom: Test the load time of that page, analyze it and find bottlenecks.
  • GTmetrix: Gives you insight on how well your site loads and provides actionable recommendations on how to optimize it.

You can see a more in-depth list on our post about website speed test tools.

Web Development Communities

web development communities

Every web developer has been there. They have a problem and what do they do? Well, they go to Google to find a quick answer. The web offers so much content right at our fingertips that it makes it easy to diagnose and troubleshoot problems when they arise. Check out a few good web development communities below.

Web Development Newsletters

web development newslettersThe last thing you probably need is another newsletter subscription. But these guys handpick some of the best web development and performance topics from around the web so you don’t have to!

  • wdrl.info: A handcrafted, carefully selected list of web development related resources. Curated and published usually every week.
  • webopsweekly.com: A weekly newsletter for Web developers focusing on web operations, infrastructure, deployment of apps, performance, and tooling, from the browser down to the metal.
  • freshbrewed.co: A weekly reading digest for front-end developers and UX designers.
  • smashingmagazine.com: Smashing Magazine is an online magazine for professional web designers and developers. Useful tips and valuable resources, sent out every second Tuesday.

And of course, you can subscribe to our newsletter if you haven’t already on the right hand side of this post.

Summary

As you can see there are hundreds of web development tools and resources available to help streamline your development workflow and hopefully aid you in being more productive. Again we can’t list every tool or resource, but if we forgot something important, feel free to let us know below in the comments.

Related Articles

100+ Awesome Web Development Tools and Resources was last modified: February 13th, 2016 by Brian Jackson


Original URL: http://feedproxy.google.com/~r/feedsapi/BwPx/~3/lUwgi_wsOqE/

Original article

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑

%d bloggers like this: