Lately, I’ve been working on a continuous integration devops dashboard for XenApp/XenDesktop, written as a SQL server database, web service and HTML5 application. The tools are evolving quickly and therefore there is a lot of out of date information on the web so I wanted to produce some notes for my colleagues which may be of wider interest.
Problem
Historically, when I wrote lots of Javascript, I’d end up with lots of SCRIPT tags to include each file. Each file would declare functions and variables, which would get jumbled up in one namespace. It could take a while for the browser to download all the script elements.
Typescript really helps produce for non-trival web applications and it is now being used routinely at Citrix on new web code. With Typescript, you catch a lot of errors as you write the code and your IDE can help you out more, so Typescript is a big productivity win over regular Javascript development.
If, however, like me, you start out not knowing what you are doing you can end up searching the web, reading a bunch of contradictory blogs, experimenting for a bit, getting confused over the 5 different kinds of Javascript modules and so on. If you aren’t careful, debugging in the browser can be tricky since you are writing in a different language that is translated to Javascript.
When I wanted to add a new module to my project, I had to:
- Download the Javascript and CSS files, maybe using node.
- Configure my web server to expose the new file by adding them to myVisual Studio projecct.
- Add a SCRIPT tag at the top of each HTML file to get the browser to load the Javascript
- Add a STYLE tag for the CSS of that mdoule.
- Download type declarations, maybe using tsd or typings.
- Instruct typescript to include the type declarations.
Solution
I use Visual Studio 2017 for development and publishing the content to IIS; the same technique should work in other environments.
Project structure
The Site project uses the ASP.Net Web Application “New Project” template. It contains some ASP.Net WebApi web service entry points plus the static HTML and CSS content for my project (The Site project also contains the database model and migrations code; I could move those into a separate probject). The Client subdirectory of the Site project contains all my typescript code.
The other projects (Propagations, Worker etc) are command line tools typically run regularly by a continuous integration system. These other projects populate the SQL database used by the Site project and produce reports. There’s no reason for such work to run in the same process or even on the same machine as the web interface and so I keep them separate. All communication to my web users goes via the database and every active thread in Site should be intiated by users, which keeps the system responsive.
Webpack
Webpack is a popular tool that takes a bunch of input material and produces the output that is needed for a number of entry points. Webpack can also produce sourcemaps, which tell the browser how to find the original source code of the input material. Typescript is well supported in Webpack (see this article on the webpack site about typescript).
Here’s my webpack.conifg.js:
/// <binding ProjectOpened='Watch - Development' />
var path = require("path");
module.exports = {
entry: {
summary : "./Site/Client/Views/Summary.ts",
buildlist: "./Site/Client/Views/BuildList.ts",
revision : "./Site/Client/Views/RevisionPage.ts"
},
output: {
filename: "[name].bundle.js",
publicPath : "/Content",
path: path.resolve(__dirname, "Site/Content")
},
devServer: {
contentBase: "Philby4",
host: "localhost",
port: 9000
},
resolve : {
extensions: ['.ts', '.js']
},
module: {
loaders: [
{
test: /\.ts?$/,
loader: "ts-loader"
}
]
},
devtool: "source-map",
externals: {
"jquery" : "jQuery",
"golden-layout" : "GoldenLayout"
}
};That’s a bit of a lump to understand and is the only tricky thing in this article. Let’s work through the key sections.
Telling webpack where to start looking for modules.
Webpack’s job (in this scenario) is to produce a bundle of javascript including a specific module and all the depedencies of that module, working recursively through each module. My application has three pages, each with its own module. So I tell webpack I want bundles of output called summary, buildlist and revision each containing a module of my code:
entry: {
summary : "./Site/Client/Views/Summary.ts",
buildlist: "./Site/Client/Views/BuildList.ts",
revision : "./Site/Client/Views/RevisionPage.ts"
},
Telling webpack where to put the bundles it makes
My ASP.Net project is set up to server static content from the root of the Site project. That’s actually a bit unfortunate, since the site static content URL structure mirrors the visual studio project. I want to keep my files organised as much as possible in subdirectories grouped by type. So I’ll put all the static content I can in a subdirectory called Content. That includes the webpack output bundles. So I tell webpack to do that by writing:
output: {
filename: "[name].bundle.js",
publicPath : "/Content",
path: path.resolve(__dirname, "Site/Content")
},Webpack needs to know how its output will be resolved when on the web server, as well as what directories to use, hence publicPath for the final web URLs and path for the current local filesystem structure. When webpack runs it will produce a Site/Content/summary.bundle.js file alongside the other 2 files.
Telling webpack to support typescript
Webpack has to be instructed to handle typescript and javascript files, and to compile typescript:
resolve : {
extensions: ['.ts', '.js']
},
module: {
loaders: [
{
test: /\.ts?$/,
loader: "ts-loader"
}
]
},
devtool: "source-map",
Telling webpack it doesn’t have to deal with absolutely everything
Webpack supports the latest syntax for ECMAScript 6 syntax (also known as ECMAScript 2015; this article helped me understand the confusing naming of Javascript variants; ECMAScript 2016 is out but didn’t add anything relevant). However many third party libraries don’t use the new syntax and do things with their names that can confuse tools that work with the new ECMAScript 6 conventions. A great thing about webpack is that you can instruct it to ignore certain modules. You’ll have to get them on to the pages yourself as you did before. So let’s do that for two libraries I use:
externals: {
"jquery" : "jQuery",
"golden-layout" : "GoldenLayout"
}
The typescript and HTML code
For 2 out of 3 pages I just have a simple static HTML file at the top level of the Site project like this:
<html> <head> <title>Philby Build List</title> <meta charset='utf-8' /> <link href='Content/Site.css' rel='stylesheet' /> <script type='text/javascript' src='http://code.jquery.com/jquery-1.11.1.min.js'> </script> <script src='https://cdn.datatables.net/1.10.11/js/jquery.dataTables.min.js'></script> <script src='Content/buildlist.bundle.js'></script> </head> <body> </body> </html>
Those 2 libraries get cached so I don’t have to handle the content through my build process, and the browser has a chance to avoid parsing it on each page load. The last script tag simply references a webpack bundle.
For the third page I did something a bit different. I don’t want my users to have to wait for the javascript to fire up and start downloading content, so I bundle it into some dynamic HTML as JSON, using WebApi:
using Newtonsoft.Json;
using Philby4.Models;
using Philby4.ViewModel;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Web.Http;
namespace Philby4.Controllers
{
public class RevisionController : ApiController
{
[Route("revision/perforce/{port}/{level}")]
public HttpResponseMessage GetRevision(int port, int level)
{
var builds = "[]";
var revision = "{}";
using (var o = new TimedOperation("get builds and description", $"for perfofce revision {level}"))
{
try
{
PerforceChange perforceChange = o.db.PerforceChanges.Where(x => x.Level == level).SingleOrDefault();
var r = new ChangeViewModel(o.db, perforceChange);
revision = JsonConvert.SerializeObject(r);
// yes, you really do have to call ToArray twice; the first call is because you can't use parameterised constructors in the
// bit of the query that goes to SQL server. The second is because you we need to do eager loading so that the construction is
// complete before we dispose of the database context inside the TimedOperation block.
var buildsDb = o.db.Builds.Where(x => x.Changes.Any(ch => ch.ChangeID == perforceChange.ChangeID)).OrderByDescending(x => x.Start).ToArray().Select(b => new BuildViewModel(b)).ToArray();
builds = JsonConvert.SerializeObject(buildsDb);
}
catch (Exception e)
{
o.SaveError(e);
throw;
}
}
var response = new HttpResponseMessage();
response.Content = new StringContent($@"<!DOCTYPE html>
<html>
<head>
<title>{level} philby data for perforce revision on {port}</title>
<link href='/Content/Site.css' rel='stylesheet' />
<script>
var globalBuilds = {builds};
var globalRevision = {revision};
</script>
<script src='/Content/revision.bundle.js'></script>
</head>
<body>
Rendering...
</body>
</html>");
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("text/html");
return response;
}
}
}In this case, the typescript code is pretty simple:
import renderRevision from '../RenderRevision';
import { ChangeViewModel } from '../../ViewModel/ChangeViewModel';
import { BuildViewModel } from '../../ViewModel/BuildViewModel';
declare var globalBuilds: BuildViewModel[];
declare var globalRevision: ChangeViewModel;
window.onload = () => {
document.body.innerHTML = renderRevision(globalBuilds, globalRevision);
}Since this code runs immedatiely in the browser it is important that we don’t import this module from another one of our mdoules otherwise the renderRevision code would start up.
(If you don’t like your modules having this kind of side effect then you can define a function, stash it in a global variable such as document, and call that from a script tag on your HTML page. I suggest putting the function into an attribute of a global variable such as document since ECMAscript 6 modules hide all their content unless you explcitily import them, and I don’t know how to import modules in code executed directly in the browser.)
Running webpack from Visual Studio 2017
You can just run webpack from powershell or command line. First install node (in the unlikely event you read this far and don’t already have it :)), then tell node to install webpack globally to your system with:npm install -g webpack ts-loader
You can then run webpack with no arguments from the solution directory where the webpack.config.js lives and it should work.
IDE users are used to not having to run commands manually to get their projects running. I don’t know how to get Visual Studio’s project build system to invoke webpack at the right time during the build. However, there’s a Visual Studio extension called webpack task runner which works well for me. I have it set up to start webpack when I open the project in a mode where webpack watches its inputs and produces the ouptut whenever the input changes (e.g. because I save a file in Visual Studio). Screenshot:
Key points
(Stuff I wish I had known a few weeks ago!)
- TSD and Typings are obsolete (they are literally months old!) but still get high rankings on google. Use “npm install @types/JavascriptFrameworkOfTheDay)” instead; see this blog from the Typescript team.
- Sourcemaps work pretty nicely when debuggning in Chrome and are easy to get going. See this article from the Chrome team. You add one line to your webpack.config.js and run Chrome on a machine that can access your project source, and then drag and drop the folder into Chrome’s developer tools. Sometimes I have to reload the page in Chrome to get the source references.
- Visual Studio 2015 and 2017 have very limited ways to control the typescript compiler compared with the full power of the compile command line interface and tsconfig.json typescript compiler config file, which Visual Studio 2015/2017 ignore (see Typescript issue 3983). Therefore, some approaches you may want to try would mean running the typescript compiler some other way. For me that meant Typescript will fail to compile in Visual Studio since you cannot get the necessary options to the compiler, which can break your build.
- Typescript does have a option –outFile (see the docs) to produce a single output file with a source map, which is similar to what webpack does . That option is supported by Visual Studio 2015 and 2017 and will work with ECMAscript 2016 modules. The reason I’m using Webpack instead is that it allows multiple output packages each with only the modules they need; it seems to me the typescript compiler would need invoking once per entry point, which is a pain.
- The project I describe here has C# with Entity Framework and ASP.Net code and Visaul Studio 2017 is good at building and debugging those. Otherwise I’d probably be using Visual Studio Code (since it is elegant, free and cross-platform and the team are doing amazing work adding useful features really quickly right now) or emacs.
- I was initally hesitant about webpack, figuring it would slow down my compile/test cycle. However, it turns out the browser is slow to load lots of files since it won’t open many HTTP requests at once and for me that saves time even factoring in the time webpack itself takes to run.
