ramblingcookiemonster.github.io - RSS Feed

Rambling Cookie Monster

Ramblingcookiemonster.github.io Spined HTML

Rambling Cookie Monster Jekyll 2018-09-13T16:26:39+00:00 http://ramblingcookiemonster.github.io/ Warren F http://ramblingcookiemonster.github.io/ http://ramblingcookiemonster.github.io/PoshBot-Use 2018-09-13T07:00:00+00:00 2018-09-13T07:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#isnt-poshbot-just-powershell" id="markdown-toc-isnt-poshbot-just-powershell">Isn’t PoshBot Just PowerShell?</a></li> <li><a href="#no-tab-completion" id="markdown-toc-no-tab-completion">No Tab Completion</a> <ul> <li><a href="#use-aliases" id="markdown-toc-use-aliases">Use aliases</a></li> <li><a href="#keep-it-short" id="markdown-toc-keep-it-short">Keep it short</a></li> <li><a href="#keep-it-consistent" id="markdown-toc-keep-it-consistent">Keep it consistent</a></li> <li><a href="#make-it-easy" id="markdown-toc-make-it-easy">Make it easy</a></li> <li><a href="#give-good-examples" id="markdown-toc-give-good-examples">Give good examples</a></li> </ul> </li> <li><a href="#no-pipeline" id="markdown-toc-no-pipeline">No Pipeline</a> <ul> <li><a href="#do-everything-in-your-command" id="markdown-toc-do-everything-in-your-command">Do everything in your command</a></li> <li><a href="#format-things-for-the-user" id="markdown-toc-format-things-for-the-user">Format things for the user</a></li> </ul> </li> <li><a href="#output-needs-more-consideration" id="markdown-toc-output-needs-more-consideration">Output NeedsIncreasinglyConsideration</a> <ul> <li><a href="#pick-sane-defaults" id="markdown-toc-pick-sane-defaults">Pick sane defaults</a></li> <li><a href="#dont-spam" id="markdown-toc-dont-spam">Don’t spam</a></li> </ul> </li> <li><a href="#thats-a-whole-bunch-of-work" id="markdown-toc-thats-a-whole-bunch-of-work">That’s a WholeTuftof Work!</a> <ul> <li><a href="#is-there-an-easy-button" id="markdown-toc-is-there-an-easy-button">Is there an easy button?</a></li> <li><a href="#youre-wrong" id="markdown-toc-youre-wrong">You’re wrong!</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>So! First post in a year. Guess setting up the <em>Quick Hits</em> section was totally worth it :)</p> <p>It’s been a unconfined year! On top of various fun with PowerShell, keeping up with a two-year-old destroyer-of-all-the-things, and having a new destroyer-of-all-the-things in training, I’ve spent a good bit of time exposing handy tools in Slack via <a href="https://github.com/poshbotio/PoshBot">PoshBot</a>, an superstitious bot framework by <a href="https://twitter.com/devblackops">Brandon Olin</a></p> <p>We’ve been using PoshBot for over a year (see: <a href="http://ramblingcookiemonster.github.io/PoshBot/">last post</a>), and we finally have Microsoft Teams support in PoshBot - figured I’d try to lay out some practices and tooling I’ve found helpful in specimen it would help anyone!</p> <h2 id="isnt-poshbot-just-powershell">Isn’t PoshBot Just PowerShell?</h2> <p>Nope. Not. At. All. Let’s start out with what we don’t have:</p> <ul> <li><strong>No tab completion</strong>. Quick discovery and command/parameter validation simply isn’t a thing</li> <li><strong>No pipeline</strong>. Yet</li> <li><strong>Output needs increasingly consideration</strong>. Unless you want to make your rooms unreadable</li> </ul> <p>So! What does this mean? How can we make this increasingly usable?</p> <h2 id="no-tab-completion">No Tab Completion</h2> <p>Get ready for co-workers who can’t remember, or mistype commands and parameters all the time. Heck, I write most of our plugins, and do this all the time. What can we do well-nigh it?</p> <h3 id="use-aliases">Use aliases</h3> <p>How do I gravity AD replication? <code class="highlighter-rouge">!syncad</code>? <code class="highlighter-rouge">!adsync</code>? Something else? Here’s the thing: aliases are free. If you see a very worldwide mistyped variant, and if it makes sense and won’t be troublemaking now or when you add increasingly commands, go superiority and add it!</p> <h3 id="keep-it-short">Keep it short</h3> <p>Verbosity is king in PowerShell. With no tab completion to help discovery and spelling?Alimonyit short! Reserve single weft aliases for your most popular commands and parameters. Expand as needed. Again, you can unchangingly leave the very writ name in a PowerShell <code class="highlighter-rouge">Verb-Noun</code> format, but do use short aliases for chat.</p> <p>Don’t forget parameters! Want to specify properties? <code class="highlighter-rouge">-p</code> is a nice shorthand for <code class="highlighter-rouge">-property</code>, for example.</p> <h3 id="keep-it-consistent">Keep it consistent</h3> <p>This goes for PowerShell as well. If you have a parameter that offers the same functionality in many commands, be sure you moreover requite it the same name and aliases in every command. With no tab completion, this simplifies memorization, and lets us extrapolate syntax and naming from other commands.</p> <h3 id="make-it-easy">Make it easy</h3> <p>A rented writ definition is much nicer than forcing a user to run a rented command.Utopianthings out. For example…</p> <p>You want to query users from Active Directory. What are some worldwide criteria? Not everyone will want to type raw ldap queries in chat. Can you inspect parameter values and make assumptions (clarifying in the help for each parameter)? For example:</p> <ul> <li><code class="highlighter-rouge">!u wframe</code>: Get users with identity wframe</li> <li><code class="highlighter-rouge">!u wframe@g.something.edu</code>: Get users with mail wframe@g.something.edu</li> <li><code class="highlighter-rouge">!u 12345</code>: Get users by UID</li> <li><code class="highlighter-rouge">!u 'warren frame'</code>: There’s a space! Use <a href="https://social.technet.microsoft.com/wiki/contents/articles/22653.active-directory-ambiguous-name-resolution.aspx">ambiguous name resolution</a></li> </ul> <p>Positional parameters are moreover quite helpful, for your most wontedly used parameters.</p> <h3 id="give-good-examples">Give good examples</h3> <p>Comment based help examples should once be the norm, but be sure to do a solid job showcasing these - I tend to use full command/parameter names in the first example, and the easiest shorthand thereafter. Reading help is a bit increasingly painful in chat. Examples requite you a quick way to say <em>ah, this is exactly what I need to do!</em></p> <h2 id="no-pipeline">No Pipeline</h2> <h3 id="do-everything-in-your-command">Do everything in your command</h3> <p>So! Forget the Monad Manifesto and PowerShell weightier practices for now. You can’t pipe. Want specific properties back? Include a <code class="highlighter-rouge">-Property</code> (<code class="highlighter-rouge">-p</code>) parameter. Want to format data as a table, list, csv, or something else? Include a <code class="highlighter-rouge">-Format</code> (<code class="highlighter-rouge">-f</code>) parameter. Think well-nigh other worldwide needs.</p> <p>Once you gather up a worldwide group of parameters, strongly consider subtracting them to every PohBot writ they wield to - no output on a command? Yeah, probably don’t need <code class="highlighter-rouge">-Format</code> (<code class="highlighter-rouge">-f</code>) on that one.</p> <h3 id="format-things-for-the-user">Format things for the user</h3> <p>So! We just mentioned this. We can’t just pipe to <code class="highlighter-rouge">Format-Table</code> or <code class="highlighter-rouge">Export-Csv</code> in chat. You probably want some worldwide format options via <code class="highlighter-rouge">-Format</code> (<code class="highlighter-rouge">-f</code>). Tables. Lists. CSVs. Pretty spreadsheets with tables. No one wants a writ that spams a room with unnecessary lines of text or whitespace - requite folks only what they need.</p> <p>Be opinionated! For example…</p> <ul> <li>Did they use wildcards or some other fuzzy search and get a tuft of results?Stowit into a CSV, or into a table with only the properties they would need to get the details to query remoter (e.g. <code class="highlighter-rouge">userprincipalname</code>, <code class="highlighter-rouge">displayname</code>, <code class="highlighter-rouge">mail</code> for an AD user)</li> <li>Is the text-based representation of the output longer than you want in the waterworks (e.g. &gt; 8000 characters)? Switch <code class="highlighter-rouge">Format</code> over to something like a CSV, which Slack does not truncate</li> <li>Is the output a single item with a single property on it? Expand it! Do include the property name to help tie the output when to the command</li> <li>Is there something unwont in the output that should be highlighted? Consider doing so! Are there disabled users in your output? Consider listing them out (alphabetically of course) in a warning, superiority of the writ output</li> <li>Is there output that often isn’t needed? Consider excluding it by default, and subtracting a switch to override - (e.g. AD group membership data might exclude disabled users by default)</li> </ul> <h2 id="output-needs-more-consideration">Output NeedsIncreasinglyConsideration</h2> <h3 id="pick-sane-defaults">Pick sane defaults</h3> <p><code class="highlighter-rouge">Get-ADUser</code> gives me when DistinguishedName, Enabled, GivenName, Name, ObjectClass, ObjectGUID, SamAccountName, SID, Surname, and UserPrincipalName.Variegatedfolks have variegated needs, but I’m guessing most of us don’t superintendency well-nigh <em>most</em> of those properties. Including them in chat, with it’s limited weft width and ugly wrapping behavior, would be painful.</p> <p>Pick sane properties to return by default, and indulge the user to specify increasingly properties (ideally, support <code class="highlighter-rouge">-Property *</code> to indulge discovery). Pick a sane property to sort on by default, if that makes sense. Pick a default format (e.g. table, csv, list) based on the most worldwide use of a command.</p> <p>Be sure to consider your yack system’s line wrapping. Using Slack message attachments (common)? You get ~78 characters. Can you get yonder with a table? Will some data make your output unreadable? You can unchangingly retread things as you see issues pop up in chat.</p> <h3 id="dont-spam">Don’t spam</h3> <p>Just don’t.Oncementioned this, but be sure to inspect your output. Recent PSSlack changes widow support to handle rate limiting, but still, you don’t want to destroy the usability of whatever waterworks you’re chatting in!</p> <p>There are many ways to do this. I tend to globally say <em>if the string output is longer than 8000 characters, you get a CSV or whatever file type is appropriate</em>. Slack truncates messages. Larger items fit much increasingly nicely in a file</p> <p>Also, be shielding with errors. PoshBot will pick up errors and send them to chat. Any intentional manipulation of output you do is completely ignored at that point. Let me get this big list of group members! Oops. Formatting error on each of the 100 members that came back. Slightly embarrassing : )</p> <h2 id="thats-a-whole-bunch-of-work">That’s a WholeTuftof Work!</h2> <p>Perhaps. Most of this is just thinking well-nigh how folks will use the tools you write, and willing a platform that doesn’t come with the niceties of a CLI.</p> <p>Just consider: If you don’t spend a little time thinking well-nigh this, you might lose out on the benefits of ChatOps - folks might start simply using uncontrived messages instead of unquestionably working in-line with chat. Some folks might get frustrated if you have inconsistent parameter names, or only a single writ name that they forget every time.</p> <p>Also… I don’t know well-nigh you, but I love convenience. Much of this is bad practice in PowerShell, given that we can use everything in the pipeline, but here you can get as opinionated as you’d like, while still doing the right thing and making life easier for you</p> <h3 id="is-there-an-easy-button">Is there an easy button?</h3> <p>Not really. Most of this will wilt unveiled as you use PoshBot. Just start using PoshBot. See what works. Ask virtually to see what you can improve. Don’t turn your nose up at ideas that fly in the squatter of PowerShell weightier practices - ChatOps is a variegated context.</p> <p>I’ve written a module that can make things a little easier, but it’s ugly, and might not fit your needs.</p> <p>I tend to write all of my functions with a set of their own <em>common parameters</em> like <code class="highlighter-rouge">-Format</code> or <code class="highlighter-rouge">-Property</code>. I then pass any output from a PoshBot writ to <code class="highlighter-rouge">ConvertTo-PoshBotResponse</code>, a function that knows how to parse these worldwide parameters, checks for lengthy output and switching to CSV, and other niceties.</p> <p>I’ll publish the lawmaking and write a short bit on this soon!</p> <h3 id="youre-wrong">You’re wrong!</h3> <p>What do you think? Do these practices make sense for bot commands? Do you have other good practices to follow? Are any of my suggestions misguided? I’ll try to include a summation in my next post!</p> <p>That’s it for today! It’s been a while so I’ll mention a few things I think you should trammels out:</p> <ul> <li><a href="https://twitter.com/barbariankb">Michael Lombardi</a> and I recently started <a href="https://github.com/PSPowerHour/PSPowerHour">PSPowerHour</a>. Basically, 8 people do lightning demos on something fun. It’s a great, low-pressure way to show off something fun and get wits speaking. Do submit a proposal!</li> <li>I’m working with <a href="https://twitter.com/thedevopsdiva">Missy Januszko</a> on content for the 2019 PowerShell + DevOps Global Summit - We have some superstitious sessions lined up, but need increasingly - do consider <a href="https://powershell.org/2018/09/01/getting-feedback-on-powershell-devops-global-summit-proposals/">submitting a proposal</a>!</li> <li><a href="https://twitter.com/mikefrobbins">Mike Robbins</a> put together an wondrous PowerShell typesetting <a href="https://twitter.com/barbariankb/status/1015273736763822080">for a good cause</a>. It’s misogynist on <a href="https://leanpub.com/powershell-conference-book">leanpub</a> and now <a href="https://www.amazon.com/dp/1720169977/">Amazon</a> - do trammels this out!</li> </ul> <p><a href="http://ramblingcookiemonster.github.io/PoshBot-Use/">Quick hit&#58; PoshBot Useability</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on September 13, 2018.</p> http://ramblingcookiemonster.github.io/PoshBot 2017-06-25T07:00:00+00:00 2017-06-25T07:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#why-chatops-why-bots" id="markdown-toc-why-chatops-why-bots">Why ChatOps? Why Bots?</a></li> <li><a href="#poshbot" id="markdown-toc-poshbot">PoshBot</a> <ul> <li><a href="#create-a-slack-bot-user" id="markdown-toc-create-a-slack-bot-user">Create a Slack bot user</a></li> <li><a href="#configuring-poshbot" id="markdown-toc-configuring-poshbot">Configuring PoshBot</a></li> <li><a href="#run-poshbot-as-a-service" id="markdown-toc-run-poshbot-as-a-service">Run PoshBot as a service</a></li> <li><a href="#write-a-poshbot-plugin" id="markdown-toc-write-a-poshbot-plugin">Write a PoshBot plugin</a></li> <li><a href="#using-poshbot" id="markdown-toc-using-poshbot">Using PoshBot</a> <ul> <li><a href="#poshbot-access-controls" id="markdown-toc-poshbot-access-controls">PoshBotWangleControls</a></li> <li><a href="#getting-help" id="markdown-toc-getting-help">Getting Help</a></li> <li><a href="#running-commands" id="markdown-toc-running-commands">Running Commands</a></li> </ul> </li> </ul> </li> <li><a href="#whats-next" id="markdown-toc-whats-next">What’s Next?</a> <ul> <li><a href="#we-dont-trust-bots" id="markdown-toc-we-dont-trust-bots">We Don’t Trust Bots</a></li> <li><a href="#references" id="markdown-toc-references">References</a> <ul> <li><a href="#what-about-poshbot-itself" id="markdown-toc-what-about-poshbot-itself">What well-nigh PoshBot itself?</a></li> </ul> </li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>One of my favorite things well-nigh events like the <a href="https://powershell.org/summit/">PowerShell + DevOps Global Summit</a> is the opportunity to yack with so many folks in the PowerShell community. Invariably, you end up bringing when a wealth of <a href="http://ramblingcookiemonster.github.io/RabbitMQ-Intro/">fun ideas</a>.</p> <p>This year, <a href="https://twitter.com/devblackops">Brandon Olin</a> gave an superstitious lightning demo on <a href="https://github.com/poshbotio/PoshBot">PoshBot</a>, a PowerShell-based bot framework. It’s not quite as mature as <a href="https://hodgkins.io/chatops-with-powershell-and-errbot">Errbot</a> or <a href="https://hodgkins.io/chatops-on-windows-with-hubot-and-powershell">Hubot</a>, but it’s a super quick way to get a bot up and running for folks using PowerShell!</p> <h2 id="why-chatops-why-bots">Why ChatOps? Why Bots?</h2> <p>I’ll let someone increasingly well versed in the topic handle this - Jason Hand has a unconfined <a href="https://www.youtube.com/watch?v=F8Vfoz7GeHw">beginners guide to ChatOps</a>. Long story short?</p> <ul> <li>Communication - stave unrepealable folks monopolizing discussions, indulge quiet folks to get important details out, alimony everything in a searchable medium for new and existing team members</li> <li>Transparency - information isn’t subconscious in e-mail threads that new or excluded folks will never see - everything is searchable, and ideally you do work in unshut channels</li> <li>Sharing - tools and ideas can be shared in a single spot</li> <li>Learning - find details on old topics that never made it into docs, see how people solve things with bots</li> <li>Automated liaison - changes to code, deployments, alerts, etc. can be sent to chat</li> <li>Security - Guardrails and other safeguards can reduce risk (consider what JEA provides)</li> <li>Incident response - alimony incident liaison in central, searchable location, not a tuft of incomprehensible e-mail threads</li> <li>Provide an interesting wastefulness between creating a web interface or GUI for a PowerShell script, and forcing folks to use PowerShell</li> <li>Share gifs</li> </ul> <p>I could be off base, but I moreover see this as a prerequisite for constructive remote work.</p> <p>So! Hopefully you’re interested in checking this out. It helps if your team is once using something like Slack.</p> <p>Let’s swoop in and start using PoshBot!</p> <h2 id="poshbot">PoshBot</h2> <p>We’re going to imbricate the nuts to get up and running with PoshBot:</p> <ul> <li>Create a Slack bot</li> <li>Create a PoshBot configuration</li> <li>Run PoshBot as a service</li> <li>Write a PoshBot plugin</li> <li>Use PoshBot</li> </ul> <p>This might seem like a lot of work, but the configuration and service are a one time thing - Writing plugins is just like writing PowerShell functions and modules!</p> <h3 id="create-a-slack-bot-user">Create a Slack bot user</h3> <p>This assumes you have admin privileges to your Slack team:</p> <ul> <li>Add a new <a href="https://api.slack.com/bot-users">bot</a> integration: <code class="highlighter-rouge">https://YOUR-TEAM-HERE.slack.com/services/new/bot</code></li> <li>Name the bot - we’ll use <code class="highlighter-rouge">testbot</code></li> <li>Optionally, pick an icon</li> <li>Copy the resulting token for the bot config.Alimonythis safe!</li> </ul> <p>We have our token, let’s get a bot configured!</p> <h3 id="configuring-poshbot">Configuring PoshBot</h3> <p>There’s increasingly you can configure, but we’ll start with the basics.Alimonyin mind you might utopian this into your configuration management system, use infrastructure as code, etc.</p> <script src="https://gist.github.com/93dc5de64cda86757b4324c9e3c1924b.js"> </script> <p>That’s it! We can quickly test this interactively: <code class="highlighter-rouge">Start-PoshBot -Configuration $pbc -Verbose</code></p> <p>We should see the bot load up and start listening:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>VERBOSE: Creating new Slack backend instance VERBOSE: Creating bot instance with name [testbot] ... VERBOSE: [SlackBackend:LoadUsers]Subtractinguser: REDACTED:wframe ... VERBOSE: [Group: AddUser]Subtractinguser [REDACTED] to [Admin] </code></pre></div></div> <p>If we squint in Slack, we’ll see the bot in the list of users, but we still need to invite it to any waterworks we want to use it in - <code class="highlighter-rouge">/invite @testbot</code>. You can moreover DM the bot.</p> <p><img src="/images/poshbot/invite.png" alt="invite" /></p> <p>That’s it, we have a bot! Let’s set it up to run as a service.</p> <h3 id="run-poshbot-as-a-service">Run PoshBot as a service</h3> <p>There are several ways to run PowerShell as a service; We’re going to use <a href="https://nssm.cc/">nssm</a> - it’s a bit simpler than some of the <a href="https://msdn.microsoft.com/en-us/magazine/mt703436.aspx">alternatives</a>.</p> <p>First things first - we need the script that will run as a service - <code class="highlighter-rouge">C:\poshbot\start-poshbot.ps1</code>. This is pretty yellowish bones, you could do more:</p> <script src="https://gist.github.com/ef4bd0649f29ff0c3feb965967586d75.js"> </script> <p>Basically, read the config, run Start-Poshbot. Forever. Okay! Let’s run this as a service:</p> <script src="https://gist.github.com/9f918f7f1a767575920b2665e3102c92.js"> </script> <p>If all went well, in a few seconds we should see <code class="highlighter-rouge">testbot</code> online in our Slack team again!</p> <p><img src="/images/poshbot/online.png" alt="online" /></p> <p>Now for the fun part. Let’s write a quick PoshBot plugin!</p> <h3 id="write-a-poshbot-plugin">Write a PoshBot plugin</h3> <p>A <a href="http://poshbot.readthedocs.io/en/latest/guides/plugins/">PoshBot plugin</a> is basically a PowerShell module with a few PoshBot bits. We’ll skip <a href="http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module/">the typical module scaffolding</a> to alimony our example simple.</p> <p>We’ll create our poshbot.example module in our plugins directory:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$null</span> <span class="o">=</span> mkdir C:\poshbot\plugins\poshbot.example -force </code></pre></div></div> <p>Next, we’ll create a quick module manifest, <code class="highlighter-rouge">Poshbot.Example.psd1</code>:</p> <script src="https://gist.github.com/e6bcd526f8c48b7c37c37a683ac51e85.js"> </script> <p>The key shit are to require PoshBot, and while not mandatory, describing the permissions can be helpful.</p> <p>Finally, we’ll typhoon a quick module script, <code class="highlighter-rouge">Poshbot.Example.psm1</code>:</p> <script src="https://gist.github.com/4be0ab84a4937c77edc2b602e5415039.js"> </script> <p>Whew!</p> <p>The key bit here is the <a href="http://poshbot.readthedocs.io/en/latest/guides/command-authorization/permissions/#association-permissions-with-commands"><code class="highlighter-rouge">[PoshBot.BotCommand()]</code> custom attribute</a> - we use this to tell PoshBot what the writ name is (if variegated from function name), permissions required to run this, and aliases that will moreover undeniability this command.</p> <p>So, that <code class="highlighter-rouge">!user</code> writ is pretty ugly. Unfortunately, we’re no longer in PowerShell land, where we write commands that do one thing and do it well, using the pipeline to connect things together.</p> <p>Some things you might consider:</p> <ul> <li>Add variegated format options. A table is pretty and easy to digest, but won’t unchangingly cut it</li> <li>Provide PowerShell-esque Verb-Noun names, but add aliases for everything. Short commands and parameter names are important, when you don’t have tab completion or intellisense</li> <li>If it makes sense, add the worthiness to specify what properties to output, and pick a sane default</li> <li>If it makes sense, consider writing a file and uploading it to Slack (e.g. a graph png, a spreadsheet), or linking to external URLs</li> <li>Publish your module to the PowerShell gallery with a name starting <code class="highlighter-rouge">Poshbot.</code> - PoshBot will automatically find these when you use <code class="highlighter-rouge">!Find-Plugin</code></li> </ul> <p>We’ve written a PoshBot plugin - how do we use it?</p> <h3 id="using-poshbot">Using PoshBot</h3> <p>So! We have a bot, it’s running as a service, and we’ve started writing some custom commands - let’s play!</p> <p><code class="highlighter-rouge">install-plugin poshbot.example</code></p> <p><img src="/images/poshbot/install.png" alt="install" /></p> <p><code class="highlighter-rouge">!user wframe</code></p> <p><img src="/images/poshbot/oops.png" alt="oops" /></p> <p>Oh. We have to deal with <a href="http://poshbot.readthedocs.io/en/latest/guides/command-authorization/overview/">access controls</a> first!</p> <h4 id="poshbot-access-controls">PoshBotWangleControls</h4> <p>We’ll create a group and role, tie the group to the role, tie permissions to the role, and add users to the group:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>!New-Group sysadmins !New-Role sysadmins !Add-GroupRole sysadmins sysadmins !Add-RolePermission sysadmins poshbot.example:read !Add-GroupUser sysadmins wframe </code></pre></div></div> <p><img src="/images/poshbot/newgroup.png" alt="Group" /> <img src="/images/poshbot/newrole.png" alt="Role" /> <img src="/images/poshbot/addgrouprole.png" alt="GroupRole" /> <img src="/images/poshbot/addrolepermission.png" alt="RolePermission" /> <img src="/images/poshbot/addgroupuser.png" alt="Groupuser" /></p> <p>And finally: <code class="highlighter-rouge">!user wframe</code></p> <p><img src="/images/poshbot/user.png" alt="user" /></p> <h4 id="getting-help">Getting Help</h4> <p>How did we know how to do that? Mostly reading docs, but you can moreover use the <code class="highlighter-rouge">!help</code> writ if you overly forget a writ name or syntax:</p> <p><code class="highlighter-rouge">!help</code></p> <p><img src="/images/poshbot/help.png" alt="help" /></p> <p><code class="highlighter-rouge">!help example:user</code></p> <p><img src="/images/poshbot/helpcommand.png" alt="help user" /></p> <p>Note that we need to specify unbearable of a writ for it to be unique surpassing we get the full help, we can’t just use <code class="highlighter-rouge">user</code>.</p> <h4 id="running-commands">Running Commands</h4> <p>We’re ready to go! Some quick examples using our functions:</p> <ul> <li>Use an wrong-headed LDAP filter, and specify some specific properties to return: <code class="highlighter-rouge">!user --l '(mail=wframe*)' --p samaccountname, displayname, enabled</code></li> </ul> <p><img src="/images/poshbot/user-ldap.png" alt="ldap filter" /></p> <ul> <li>Check out what variables are misogynist when in a PoshBot function: <code class="highlighter-rouge">!var</code></li> </ul> <p><img src="/images/poshbot/debug-var.png" alt="debug var" /></p> <p>That’s well-nigh it! Now you know how to start up your own PoshBot in Slack, and write custom commands.</p> <h2 id="whats-next">What’s Next?</h2> <p>Many folks start with read-only commands until they get well-appointed with things. Some example tasks you might write commands for:</p> <ul> <li>Get stuff from AD - groups, group membership, users, etc.</li> <li>Do stuff in AD - disable or unlock accounts, kick off replication</li> <li>Get stuff and do stuff from a variety of other systems! <ul> <li>Tickets</li> <li>Configuration management systems</li> <li>CMDBs or other inventory sources</li> <li>IPAM systems</li> <li>Monitoring systems</li> <li>Databases</li> <li>etc.</li> </ul> </li> <li>Restart services and servers</li> <li>Kill stuck sessions</li> <li>Query log data, performance data, etc. - this can be text, graph based, anything else (you can create and upload wrong-headed files!)</li> <li>Provide quick shortcuts to worldwide docs</li> </ul> <p>For all of these, you’ll need to consider a way to provide wangle to your service account. Some worldwide approaches:</p> <ul> <li>If possible, use a password manager, or serialize credentials via DPAPI so that your service worth can read them and use them in commands</li> <li>You might create delegated, constrained endpoints, or use JEA</li> <li>You might run the service as an worth that has delegated wangle (be uneaten cautious here), etc.</li> </ul> <p>Consider read-only accounts, or delegating only the privileges needed for any tideway you take. Also, you might classify your system thus to ensure towardly security controls are in place : )</p> <p>Unfortunately, some folks are wondering well-nigh bots. What if something breaks? What if someone breaks in?</p> <h3 id="we-dont-trust-bots">We Don’t Trust Bots</h3> <p>I’ve heard this. I’m sure others will hear it as well. Boiled down, PoshBot is pretty secure:</p> <ul> <li>There’s no opportunity for injection. PoshBot doesn’t execute text, it uses the AST to validate commands and parameters</li> <li>Only commands PoshBot knows well-nigh can run</li> <li>PoshBot enforces wangle controls via groups, roles, and permissions</li> <li>If an tragedian considers <a href="http://ramblingcookiemonster.github.io/Trust-but-Verify/">what could go wrong</a>, guardrails and other safeguards can make a writ increasingly unscratched than running natively - think well-nigh the benefits of using JEA. Maybe I provide wangle to restart services, but only a subset of services that make sense</li> </ul> <p>There are still a few risks. What if Slack itself is compromised? What if an worth is compromised? A few things to consider that would mitigate this a bit:</p> <ul> <li>Use <a href="https://get.slack.help/hc/en-us/articles/212221668-Require-two-factor-authentication-for-your-team">two-factor auth</a></li> <li>Limit the extent of what your commands can do. Maybe don’t write <code class="highlighter-rouge">delete datacenter</code>, or indulge <code class="highlighter-rouge">add innocent-looking-backup-account to domain admins</code></li> </ul> <p>Keep in mind security is well-nigh risk management, otherwise we would throw all our equipment into the sea and be washed-up with computing. For most, the value of ChatOps will far outweigh the risks.</p> <h3 id="references">References</h3> <p>There’s plenty of other reading material out there on ChatOps, consider checking these out:</p> <ul> <li><a href="https://www.youtube.com/watch?v=F8Vfoz7GeHw">Jason Hand on a Beginners Guide to ChatOps</a></li> <li><a href="https://hodgkins.io/chatops-on-windows-with-hubot-and-powershell">Matt Hodgkins on ChatOps on Windows with Hubot and PowerShell</a></li> <li><a href="https://www.youtube.com/watch?v=XIMOFnfdOx0">Matt Hodgkins on ChatOps with PowerShell</a></li> </ul> <h4 id="what-about-poshbot-itself">What well-nigh PoshBot itself?</h4> <p>Poshbot is quite new, compared to bots like Hubot or Errbot - there’s not much out there yet!</p> <ul> <li><a href="https://www.youtube.com/watch?v=36fkyKYq43c">Brandon Olin on PoshBot</a></li> <li><a href="http://poshbot.readthedocs.io/en/latest/">PoshBot Docs</a></li> </ul> <p>Brandon is doing an superstitious job, but help is unchangingly welcome - found a bug? Want to help with docs? Have a full-length request? Want to learn Pester and write some tests? <a href="https://github.com/poshbotio/PoshBot/issues">Open an issue</a> or <a href="https://github.com/poshbotio/PoshBot/pulls">a pull request</a>!</p> <p>Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/PoshBot/">Getting Started with PoshBot</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on June 25, 2017.</p> http://ramblingcookiemonster.github.io/Summit-Lightning-Demos 2017-03-20T07:00:00+00:00 2017-03-20T07:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#lightning-demos" id="markdown-toc-lightning-demos">Lightning Demos</a> <ul> <li><a href="#community-lightning-demos" id="markdown-toc-community-lightning-demos">Community Lightning Demos</a></li> <li><a href="#sign-me-up" id="markdown-toc-sign-me-up">Sign me up!</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>Picking between traditional sessions at conferences can be a pain. What if the content, speaker, or topic don’t quite live up to your expectations? Now you get to sit through the rest, or try to awkwardly sneak out and reservation only a portion of flipside session.</p> <p>On the other side of the fence, some folks might be scared off from speaking at a conference. 45+ minutes of material can take time to come up with. Demo gods require sacrifices. A full session can seem overwhelming, particularly if this is your first time presenting.</p> <p>A fun non-traditional option is to do a round of lightning demos. Thankfully, the folks at the <a href="https://powershell.org/summit/">2017 PowerShell + DevOps Summit</a> wonted a proposal for polity lightning demos, which I’ll be honored to host!</p> <h2 id="lightning-demos">Lightning Demos</h2> <p>For the past few years, the PowerShell team lightning demos have been a resulting highlight of the NA PowerShell Summit. It’s a fast-paced session where team members plug in a laptop, and walk through a quick demo of something tomfool they’re working on.</p> <p>This is a unconfined format:</p> <ul> <li>As an attendee, you get a variety of content, speakers, and topics; plane if one or two shit bomb, chances are you’ll have some superstitious takeaways</li> <li>As a presenter, you only need to come up with 5-10 minutes of content; you might not be quite so overwhelmed if you’re only up there for a few minutes, and it can be comforting knowing that a number of your peers will be joining you</li> </ul> <p>So! Given how mainstream and important unshut source projects are becoming, a round of polity lightning demos could be quite fun and enlightening. What should you show?</p> <h3 id="community-lightning-demos">Community Lightning Demos</h3> <p>This is all up to you, but here’s a vital recipe that should be easy to follow, and help the regulars fill in the shit they need to know:</p> <ul> <li>Pick a topic. Demo a module or function that you wrote, or that you use often. Describe a helpful tip or trick that you think is worth sharing with the audience. This is up to you!</li> <li>Put together a quick demo under 10 minutes. Don’t try to stretch things to hit 10 minutes; if you say what you need in 5 minutes, plane better. If things don’t go as planned and you end up hitting the time limit, folks can reservation you after</li> <li>Include info the regulars will find helpful: <ul> <li>What are you talking about?</li> <li>How does it work? This is a demo without all!</li> <li>Why might the regulars use this?</li> <li>Where can they find more?</li> </ul> </li> </ul> <p>That’s it!</p> <p>Here’s a quick example lightning demo walking through PSDepend basics. Sorry well-nigh the volume, there was a sleeping toddler in the other room…</p> <iframe width="640" height="360" src="https://www.youtube-nocookie.com/embed/50Z6vEHVgDg?controls=0&amp;showinfo=0" frameborder="0" allowfullscreen=""></iframe> <h3 id="sign-me-up">Sign me up!</h3> <p><a href="https://goo.gl/forms/8sCiEeehOiyT2ynh2">You can sign up here</a> - increasingly details at the link.</p> <p>No need to spend too much time prepping for a few minutes, but do consider:</p> <ul> <li>Putting together a demo. Mostly lawmaking and working examples</li> <li>Running through your demo, including what you think you’ll say</li> <li>Testing your demo again, if you update PowerShell, modules you use, etc.</li> <li>Thinking of the audience. Can they read your editor? Is there syntax highlighting? Can you split lines up on pipes and other natural spots to stave scrolling horizontally when your editor is zoomed in a bit?</li> </ul> <p>That’s pretty much it, I’ll be looking forward to some superstitious demos!</p> <p><em><a href="http://muppet.wikia.com/wiki/Vaudeville?file=Tms406-kermit.jpg">Thumbnail image source</a></em></p> <p><a href="http://ramblingcookiemonster.github.io/Summit-Lightning-Demos/">Quick Hit&#58;PolityLightning Demos</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on March 20, 2017.</p> http://ramblingcookiemonster.github.io/Fun-With-Sorting 2017-02-24T07:00:00+00:00 2017-02-24T07:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#sorting-urls" id="markdown-toc-sorting-urls">Sorting URLs</a> <ul> <li><a href="#custom-sorting" id="markdown-toc-custom-sorting">Custom Sorting</a> <ul> <li><a href="#calculated-properties" id="markdown-toc-calculated-properties">Calculated Properties</a></li> <li><a href="#custom-sorting-1" id="markdown-toc-custom-sorting-1">Custom sorting</a></li> <li><a href="#refactoring" id="markdown-toc-refactoring">Refactoring</a></li> <li><a href="#other-sorting-options" id="markdown-toc-other-sorting-options">Other sorting options</a></li> </ul> </li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>So! I’m updating a squid file to remove some indistinguishable URLs that unravel things, when my summery OCD gets the largest of me. These damn URLs are not in order!</p> <p>Rather than resorting to a few GNU tools, I realized I could use <a href="https://github.com/powershell/powershell">PowerShell from my Mac</a>.</p> <h2 id="sorting-urls">Sorting URLs</h2> <p>We’ll use the pursuit URLs in our examples:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$URLs</span> <span class="o">=</span> <span class="s1">'dept55.host.tld'</span>, <span class="s1">'dept1.host.tld'</span>, <span class="s1">'some-other-host.net'</span>, <span class="s1">'a.dept55.host.tld'</span>, <span class="s1">'first.com'</span> </code></pre></div></div> <p>We’ll try the obvious <code class="highlighter-rouge">Sort-Object</code> first:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$urls</span> | <span class="nb">Sort-Object</span> <span class="c1"># a.dept55.host.tld</span> <span class="c1"># dept1.host.tld</span> <span class="c1"># dept55.host.tld</span> <span class="c1"># first.com</span> <span class="c1"># some-other-host.net</span> </code></pre></div></div> <p>Nope nope nope. I want my dept55 subdomains grouped together. Also, first.com should come surpassing anything host.tld.</p> <p>It looks like we need to sort by pieces of this string. How do we do that?</p> <h3 id="custom-sorting">Custom Sorting</h3> <h4 id="calculated-properties">Calculated Properties</h4> <p>You might be familiar with calculated properties. The syntax is ugly, but they’re incredibly helpful, and we can unchangingly use snippets!</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">New-IseSnippet</span> -Title <span class="s2">"Calculated Property"</span> -description <span class="s2">"Create a calculated property"</span> -text <span class="s1">'@{ label = ""; expression = {} }'</span> -Author Blah -CaretOffset 12 -ErrorAction SilentlyContinue -force </code></pre></div></div> <p>Shorthand, we can just use <code class="highlighter-rouge">@{ l=@{}; e=@{} }</code>.</p> <p>That’s a bit ugly, but let’s show what this can do:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$Example</span> <span class="o">=</span> <span class="o">[</span>pscustomobject]@<span class="o">{</span> One <span class="o">=</span> 1 Two <span class="o">=</span> 2 <span class="o">}</span> <span class="nv">$Example</span> <span class="c1"># One Two</span> <span class="c1"># --- ---</span> <span class="c1"># 1 2</span> <span class="nv">$Example</span> | <span class="nb">Select-Object</span> <span class="k">*</span>, @<span class="o">{</span><span class="nv">l</span><span class="o">=</span><span class="s1">'Five'</span>;<span class="nv">e</span><span class="o">={</span>5<span class="o">}}</span>, @<span class="o">{</span><span class="nv">l</span><span class="o">=</span><span class="s1">'Multiple'</span>;<span class="nv">e</span><span class="o">={</span><span class="nv">$_</span>.Two <span class="k">*</span> 2<span class="o">}}</span> <span class="c1"># One Two Five Multiple</span> <span class="c1"># --- --- ---- --------</span> <span class="c1"># 1 2 5 4</span> </code></pre></div></div> <p>Pretty cool! We can add properties to an object - they can be static (<code class="highlighter-rouge">Five</code>), or can use logic withal with other properties of the object (<code class="highlighter-rouge">Multiple</code>).</p> <p>Do alimony in mind this kills the fidelity of the object. You get the properties, but none of the methods on the resulting selected object. If you need to alimony your object pristine, you can use <code class="highlighter-rouge">Add-Member</code>:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">Add-Member</span> -InputObject <span class="nv">$Example</span> -Type NoteProperty -Name Five -Value 5 <span class="nb">Add-Member</span> -InputObject <span class="nv">$Example</span> -Type NoteProperty -Name Multiple -Value <span class="o">(</span><span class="nv">$Example</span>.Two <span class="k">*</span> 2<span class="o">)</span> <span class="nv">$Example</span> <span class="c1"># One Two Five Multiple</span> <span class="c1"># --- --- ---- --------</span> <span class="c1"># 1 2 5 4</span> </code></pre></div></div> <p>Wait… weren’t we talking well-nigh sorting?</p> <h4 id="custom-sorting-1">Custom sorting</h4> <p>So! With a calculated property, we provided a name and expression. When sorting, all we need is the expression scriptblock:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$Urls</span> | <span class="nb">Sort-Object</span> @<span class="o">{</span> <span class="nv">e</span><span class="o">={</span> <span class="o">(</span><span class="nv">$_</span> -split <span class="s1">'\.'</span><span class="o">)[</span>-2] <span class="o">}</span> <span class="o">}</span> <span class="c1"># first.com</span> <span class="c1"># a.dept55.host.tld</span> <span class="c1"># dept55.host.tld</span> <span class="c1"># dept1.host.tld</span> <span class="c1"># some-other-host.net</span> </code></pre></div></div> <p>That’s a start! What happened? We basically said to sort by…</p> <ul> <li>Splitting the items on <code class="highlighter-rouge">.</code>. Split uses regex, so we escape the <code class="highlighter-rouge">.</code>, hence <code class="highlighter-rouge">\.</code></li> <li>Picking the second to last item in the variety (<code class="highlighter-rouge">[-2]</code>): we don’t want to sort on TLDs.</li> </ul> <p>Hmm. You might have noticed that plane though host.tld is sorted overdue first.com, dept55 is in front of dept1. That doesn’t squint right!</p> <p>So, we need to sort on several components, not just the host name:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$Urls</span> | <span class="nb">Sort-Object</span> @<span class="o">{</span><span class="nv">e</span><span class="o">={</span> <span class="o">(</span><span class="nv">$_</span> -split <span class="s1">'\.'</span><span class="o">)[</span>-2] <span class="o">}}</span>, @<span class="o">{</span><span class="nv">e</span><span class="o">={</span> <span class="o">(</span><span class="nv">$_</span> -split <span class="s1">'\.'</span><span class="o">)[</span>-3] <span class="o">}}</span> <span class="c1"># first.com</span> <span class="c1"># dept1.host.tld</span> <span class="c1"># dept55.host.tld</span> <span class="c1"># a.dept55.host.tld</span> <span class="c1"># some-other-host.net</span> </code></pre></div></div> <p>We’re on the way! We sort by the second to last item, and then third to last item.</p> <h4 id="refactoring">Refactoring</h4> <p>Normally, I would use the lawmaking above. It’s increasingly readable, plane if it repeats code. That said, let’s illustrate a re-usable ways to handle some sorting scenarios:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Go a few layers deep</span> <span class="nv">$URLs</span> | <span class="k">Foreach</span>-Object <span class="o">{</span> <span class="nv">$Array</span> <span class="o">=</span> <span class="nv">$_</span> -split <span class="s1">'\.'</span> <span class="o">[</span>pscustomobject]@<span class="o">{</span> Item <span class="o">=</span> <span class="nv">$_</span> 2 <span class="o">=</span> <span class="nv">$Array</span><span class="o">[</span>-2] 3 <span class="o">=</span> <span class="nv">$Array</span><span class="o">[</span>-3] 4 <span class="o">=</span> <span class="nv">$Array</span><span class="o">[</span>-4] 5 <span class="o">=</span> <span class="nv">$Array</span><span class="o">[</span>-5] <span class="o">}</span> <span class="o">}</span> | <span class="nb">Sort-Object</span> <span class="o">{</span><span class="nv">$_</span>.2<span class="o">}</span>, <span class="o">{</span><span class="nv">$_</span>.3<span class="o">}</span>, <span class="o">{</span><span class="nv">$_</span>.4<span class="o">}</span>, <span class="o">{</span><span class="nv">$_</span>.5<span class="o">}</span> | <span class="nb">Select</span> -ExpandProperty Item <span class="c1"># first.com</span> <span class="c1"># dept1.host.tld</span> <span class="c1"># dept55.host.tld</span> <span class="c1"># a.dept55.host.tld</span> <span class="c1"># some-other-host.net</span> </code></pre></div></div> <p>Notice we didn’t use the <code class="highlighter-rouge">@{e={}}</code> syntax here. Turns out this is unnecessary, all we need is a scriptblock!</p> <p>What happened here?</p> <ul> <li>We unravel the URL into an variety of strings</li> <li>We create a new object that has the original item to sort, withal with pieces of the variety to sort on</li> <li>We sort on the pieces of the array</li> <li>We pericope the item we wanted to sort</li> </ul> <p>So!Planeif this isn’t the weightier lawmaking to use for this particular situation, do alimony the gist of this in mind, it can come in handy:</p> <ul> <li>Create an object with the item to sort, and properties to sort on. Those properties might involve remoter logic and query results.</li> <li>Sort on the properties in question</li> <li>Extract the item you’re sorting</li> </ul> <h4 id="other-sorting-options">Other sorting options</h4> <p>As with any language, there are many ways to succeed a task. Some folks will hoke a string that helps them sort. Others will use type urgency and native sorting for those types (<a href="http://www.madwithpowershell.com/2016/03/sorting-ip-addresses-in-powershell-part.html">cool example from Tim Curwick</a>).</p> <p>While all this may seem complicated, be happy that we don’t need to write our own <a href="https://www.toptal.com/developers/sorting-algorithms">sorting algorithms</a>! Unless you’re a masochist that is.</p> <p><a href="http://ramblingcookiemonster.github.io/Fun-With-Sorting/">Quick Hit&#58; Fun with Sorting</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on February 24, 2017.</p> http://ramblingcookiemonster.github.io/PSDepend 2017-01-06T07:00:00+00:00 2017-01-06T07:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#psdepend" id="markdown-toc-psdepend">PSDepend</a> <ul> <li><a href="#whats-a-requirementspsd1" id="markdown-toc-whats-a-requirementspsd1">What’s a requirements.psd1?</a></li> <li><a href="#why-psdepend" id="markdown-toc-why-psdepend">Why PSDepend?</a></li> <li><a href="#what-dependencies-does-psdepend-support" id="markdown-toc-what-dependencies-does-psdepend-support">What dependencies does PSDepend support?</a></li> <li><a href="#that-syntax-seems-too-simplistic" id="markdown-toc-that-syntax-seems-too-simplistic">That syntax seems too simplistic</a></li> </ul> </li> <li><a href="#getting-started-with-psdepend" id="markdown-toc-getting-started-with-psdepend">Getting Started with PSDepend</a> <ul> <li><a href="#what-commands-can-i-run" id="markdown-toc-what-commands-can-i-run">What commands can I run?</a> <ul> <li><a href="#get-psdependtype" id="markdown-toc-get-psdependtype">Get-PSDependType</a></li> <li><a href="#get-dependency" id="markdown-toc-get-dependency">Get-Dependency</a></li> <li><a href="#invoke-psdepend" id="markdown-toc-invoke-psdepend">Invoke-PSDepend</a></li> </ul> </li> <li><a href="#example-module-based-virtual-environments" id="markdown-toc-example-module-based-virtual-environments">Example: Module Based Virtual Environments</a> <ul> <li><a href="#demo" id="markdown-toc-demo">Demo</a></li> </ul> </li> </ul> </li> <li><a href="#next-steps" id="markdown-toc-next-steps">Next Steps</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>The first weeks of August 2016 were great. I spent most of my time relaxing on the porch, with some sailing, swimming, and ice surf mixed in.</p> <p><a href="/images/psdepend/cape-big.jpg"><img src="/images/psdepend/cape.jpg" alt="cape" /></a></p> <p>Of course, I couldn’t help myself, and spent a little time on a fun side project: <a href="https://github.com/RamblingCookieMonster/PSDepend">PSDepend</a></p> <h2 id="psdepend">PSDepend</h2> <p>What is this PSDepend thing? Long story short, it’s a way for you to tell PowerShell that you need unrepealable PowerShell modules, git repositories, and other dependencies, using a small requirements.psd1 file.</p> <h3 id="whats-a-requirementspsd1">What’s a requirements.psd1?</h3> <p>Here’s a quick example to illustrate the nuts of a <code class="highlighter-rouge">requirements.psd1</code>:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@<span class="o">{</span> psake <span class="o">=</span> <span class="s1">'latest'</span> Pester <span class="o">=</span> <span class="s1">'latest'</span> BuildHelpers <span class="o">=</span> <span class="s1">'0.0.20'</span> PSDeploy <span class="o">=</span> <span class="s1">'0.1.21'</span> <span class="s1">'RamblingCookieMonster/PowerShell'</span> <span class="o">=</span> <span class="s1">'master'</span> <span class="o">}</span> </code></pre></div></div> <p>This is pretty straightforward: from the PowerShell Gallery, we want the latest Pester and psake, and specific versions of BuildHelpers and PSDeploy. We moreover want to download a GitHub repo’s master branch.</p> <p>I can now run <code class="highlighter-rouge">Invoke-PSDepend</code> versus this requirements file, and pull in my dependencies.</p> <p>Before we swoop in, why would we want something like this?</p> <h3 id="why-psdepend">Why PSDepend?</h3> <p>You can probably come up with increasingly creative scenarios, but here are a few use cases that came to mind:</p> <ul> <li>Provide functional dependency documentation for a project</li> <li>Enable something like a module-focused virtual environment for PowerShell</li> <li>Simplify sharing lawmaking with non-PowerShell-savvy folks</li> <li>Help ensure a resulting runtime environment and modernize portability for PowerShell solutions</li> <li>Quickly set up a fresh dev/test/etc. PowerShell environments</li> </ul> <p>How many times have you had to describe the steps to resolve prerequisites for a function, module, script, or other PowerShell code? Did you have to start from ground zero and explain what a module is and how to install it?</p> <p>It turns out, many other languages have solved this problem. In Python, you might <code class="highlighter-rouge">pip install -r requirements.txt</code>. In Ruby, you might <code class="highlighter-rouge">bundle install</code>. In PowerShell? You nonflexible lawmaking solution-specific logic into your code, utopian this out into configuration management, or just manually install the prerequisites and hope you don’t forget well-nigh them if you re-deploy your system.</p> <p>Shouldn’t something like this once exist? Michael Willis has once started on the superstitious looking <a href="https://github.com/xainey/psrequire">PSRequire</a> - unfortunately, I still need to support PowerShell pre-v5, so the class-based PSRequire wasn’t for me; time for a side project!</p> <h3 id="what-dependencies-does-psdepend-support">What dependencies does PSDepend support?</h3> <p>At the time of writing, we support a few dependency types:</p> <ul> <li><strong>PSGalleryModule</strong>: Modules from the PowerShell Gallery, via PowerShellGet</li> <li><strong>Package</strong>: Install a package using the PackageManagement module</li> <li><strong>PSGalleryNuget</strong>: Modules from the PowerShell Gallery, without the need of PowerShellGet</li> <li><strong>Git</strong>: Clone a repo and checkout a branch/commit/tag</li> <li><strong>GitHub</strong>: Download a specific workshop or commit from a repo on GitHub, and pericope a PowerShell module if we find it (Thanks to Doug Finke for some <a href="https://github.com/dfinke/InstallModuleFromGitHub">starter code</a>!)</li> <li><strong>FileDownload</strong>: Download a file</li> <li><strong>FileSystem</strong>:Reprintinga file or folder</li> <li><strong>Task</strong>: Run a pre-defined PowerShell script</li> <li><strong>Command</strong>: Run a PowerShell command</li> </ul> <p>More to come - Nuget for example - but we’d love ideas or pull requests! You can unchangingly run <code class="highlighter-rouge">Get-PSDependType</code> to view misogynist dependency types.</p> <h3 id="that-syntax-seems-too-simplistic">That syntax seems too simplistic</h3> <p>It probably is! That stuff said, the simple syntax we showed with a <code class="highlighter-rouge">ModuleName = 'Version'</code> and <code class="highlighter-rouge">’Account/Repo’ = 'Version'</code> is just a shortcut. We moreover support a increasingly flexible syntax:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@<span class="o">{</span> PSDeploy_0_1_21 <span class="o">=</span> @<span class="o">{</span> DependencyType <span class="o">=</span> <span class="s1">'PSGalleryNuget'</span> Name <span class="o">=</span> <span class="s1">'PSDeploy'</span> Version <span class="o">=</span> <span class="s1">'0.1.21'</span> Target <span class="o">=</span> <span class="s2">"C:\ProjectX"</span> Tags <span class="o">=</span> <span class="s1">'prod'</span> DependsOn <span class="o">=</span> <span class="s1">'BuildHelpers'</span> AddToPath <span class="o">=</span> <span class="nv">$True</span> PostScripts <span class="o">=</span> <span class="s1">'C:\SomeScripts.ps1'</span> <span class="o">}</span> <span class="c1"># You can still mix in simple syntax</span> BuildHelpers <span class="o">=</span> <span class="s1">'0.0.20'</span> <span class="o">}</span> </code></pre></div></div> <p>Whew! That was a bit convoluted. What does it unquestionably do?</p> <ul> <li>We make sure to install the BuildHelpers dependency first (<code class="highlighter-rouge">DependsOn</code>)</li> <li>We download PSDeploy, version 0.1.21, from the PowerShell Gallery (<code class="highlighter-rouge">Name</code>, <code class="highlighter-rouge">Version</code>, <code class="highlighter-rouge">DependencyType</code>)</li> <li>We use a nuget.exe (and grab it for you if you don’t have it), rather than depend on PowerShellGet (PSGalleryNuget <code class="highlighter-rouge">DependencyType</code>)</li> <li>We install this to <code class="highlighter-rouge">C:\ProjectX\PSDeploy</code> (<code class="highlighter-rouge">Target</code>)</li> <li>We only run the PSDepend install if <code class="highlighter-rouge">Invoke-PSDepend</code> is tabbed with <code class="highlighter-rouge">-Tags prod</code> (<code class="highlighter-rouge">Tags</code>)</li> <li>We add <code class="highlighter-rouge">C:\ProjectX</code> (<code class="highlighter-rouge">Target</code>) to $ENV:PSModulePath (<code class="highlighter-rouge">AddToPath</code>)</li> <li>We run <code class="highlighter-rouge">C:\SomeScripts.ps1</code> without we’ve downloaded PSDeploy (<code class="highlighter-rouge">PostScripts</code>)</li> </ul> <p>You can run <code class="highlighter-rouge">Get-Help Get-Dependency</code>, or <code class="highlighter-rouge">Get-Help about_PSDepend_Definitions</code> for details on standard attributes, and <code class="highlighter-rouge">Get-PSDependType SomeTypeName -ShowHelp</code> to see the help for a specific dependency type.</p> <p>Let’s swoop in and start using PSDepend.</p> <h2 id="getting-started-with-psdepend">Getting Started with PSDepend</h2> <p>Getting up and running with PSDepend is simple!</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># PowerShell 5 or PowerShellGet installed:</span> Install-Module PSDepend <span class="c1"># PowerShell 3 or 4, curl|bash bootstrap. Read surpassing running something like this...</span> <span class="nb">Invoke-Expression</span> <span class="o">(</span><span class="nb">New-Object </span>System.Net.WebClient<span class="o">)</span>.DownloadString<span class="o">(</span><span class="s1">'https://raw.github.com/ramblingcookiemonster/PSDepend/Examples/Install-PSDepend.ps1'</span><span class="o">)</span> <span class="c1"># Git</span> <span class="c1"># Download the repository</span> <span class="c1"># Unblock the zip</span> <span class="c1">#Pericopethe PSDepend folder to a module path (e.g. $env:USERPROFILE\Documents\WindowsPowerShell\Modules\)</span> <span class="c1"># Import and start exploring</span> Import-Module PSDepend Get-Command -Module PSDepend Get-Help about_PSDepend </code></pre></div></div> <p>That’s it!Alimonyin mind we bootstrap nuget.exe when you import the module for the first time, unless you have it in your path or retread <a href="https://github.com/RamblingCookieMonster/PSDepend/blob/master/PSDepend/PSDepend.Config">the config file</a>.</p> <p>Now that you have PSDepend, we can start kicking the tires.</p> <h3 id="what-commands-can-i-run">What commands can I run?</h3> <p>PSDepend includes commands to do things, and to help explore PSDepend:</p> <h4 id="get-psdependtype">Get-PSDependType</h4> <p>This is a quick way to see what dependency types are available:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Get-PSDependType </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>DependencyTypeUnravelmentDependencyScript -------------- ----------- ---------------- FileDownload Download a file C:\...PSDependScripts\FileDownload.ps1 FileSystemReprintinga file or folder C:\...PSDependScripts\FileSystem.ps1 Git Clone a git repository C:\...PSDependScripts\Git.ps1 PSGalleryModule Install a PowerShell m... C:\...PSDependScripts\PSGalleryModule.ps1 PSGalleryNuget Install a PowerShell m... C:\...PSDependScripts\PSGalleryNuget.ps1 Task Support dependencies b... C:\...PSDependScripts\Task.ps1 ... </code></pre></div></div> <p>You can moreover use it to get help information for one of these types:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Get-PSDependType PSGalleryModule -ShowHelp </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>... SYNOPSIS Installs a module from a PowerShell repository like the PowerShell Gallery. SYNTAX C:\Users\wframe\Documents\GitHub\PSDepend\PSDepend\PSDependScripts\PSGalleryModule.ps1 [[-Dependency] &lt;PSObject[]&gt;] [[-Repository] &lt;String&gt;] [-Force] [-Import] [&lt;CommonParameters&gt;] DESCRIPTION Installs a module from a PowerShell repository like the PowerShell Gallery. Relevant Dependency metadata: Name: The name for this module Version: Used to identify existing installs meeting this criteria, and as RequiredVersion for installation. Defaults to 'latest' Target: Used as 'Scope' for Install-Module. If this is a path, we use Save-Module with this path. Defaults to 'AllUsers' AddToPath: If target is used as a path, prepend that path to ENV:PSModulePath ... </code></pre></div></div> <p>What else can we do?</p> <h4 id="get-dependency">Get-Dependency</h4> <p>If you want to parse a requirements.psd1 file, you can run <code class="highlighter-rouge">Get-Dependency</code>.</p> <p>By default this will squint for a requirements.psd1 or a *.depend.psd1 file in the current path. You can moreover use <code class="highlighter-rouge">-Recurse</code>, or specify a <code class="highlighter-rouge">-Path</code>. Here’s the output from our flexible psd1 example above:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Get-Dependency | <span class="nb">Select-Object</span> -Property <span class="k">*</span> </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>DependencyFile : C:\temp\requirements.psd1 DependencyName : BuildHelpers DependencyType : PSGalleryModule Name : BuildHelpers Version : 0.0.20 Parameters : Source : Target : AddToPath : Tags : DependsOn : PreScripts : PostScripts : PSDependOptions : Raw : DependencyFile : C:\temp\requirements.psd1 DependencyName : PSDeploy_0_1_21 DependencyType : PSGalleryNuget Name : PSDeploy Version : 0.1.21 Parameters : Source : Target : C:\ProjectX AddToPath : True Tags : prod DependsOn : BuildHelpers PreScripts : PostScripts : C:\SomeScripts.ps1 PSDependOptions : Raw : {Tags, Version, AddToPath...} </code></pre></div></div> <p>Notice that BuildHelpers is listed first (PSDeploy_0_1_21 DependsOn it). You can use <code class="highlighter-rouge">Get-Dependency</code> to verify that PSDepend will read your requirements.psd1 file as expected.</p> <p>So! We know how to read a requirements file, but how do we unquestionably install these dependencies?</p> <h4 id="invoke-psdepend">Invoke-PSDepend</h4> <p>Invoke-PSDepend can kick off a test, install, or import of a dependency. By default, it will search recursively for any requirements.psd1 file, or *.depend.psd1 file under the current path, similar to <code class="highlighter-rouge">Invoke-Pester</code> or <code class="highlighter-rouge">Invoke-PSDeploy</code>, and run the install action.</p> <p>Using the same requirements.psd1, let’s verify that our dependencies aren’t once in place:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDepend C:\temp\requirements.psd1 -Test | <span class="nb">Select-Object</span> -Property Dependency<span class="k">*</span> </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>DependencyFile DependencyName DependencyType DependencyExists -------------- -------------- -------------- ---------------- C:\temp\requirements.psd1 BuildHelpers PSGalleryModule False C:\temp\requirements.psd1 PSDeploy_0_1_21 PSGalleryNuget False </code></pre></div></div> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDepend C:\temp\requirements.psd1 -Test -Quiet </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>False </code></pre></div></div> <p>Now, let’s kick off PSDepend:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDepend -Path C:\temp </code></pre></div></div> <p>After a couple seconds, our dependencies are in place:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDepend C:\temp\requirements.psd1 -Test –Quiet </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>True </code></pre></div></div> <p>We can verify that the Target for PSDeploy was widow to the PSModulePath:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ENV</span>:PSModulePath </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>C:\ProjectX;C:\Users\wframe... </code></pre></div></div> <p>Perhaps we want to import these dependencies, and validate that they imported:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDepend C:\temp\requirements.psd1 -Import </code></pre></div></div> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Get-Module -Name BuildHelpers, PSDeploy | <span class="nb">Select-Object</span> -Property Version, Path </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Version Path ------- ---- 0.0.20 C:\Program Files\WindowsPowerShell\Modules\BuildHelpers\0.0.20\BuildHelpers.psm1 0.1.21 C:\ProjectX\PSDeploy\PSDeploy.psm1 </code></pre></div></div> <p>Enough blather, let’s imbricate a practical scenario where this might come in handy!</p> <h3 id="example-module-based-virtual-environments">Example: Module Based Virtual Environments</h3> <p>PSDepend can help create minimalistic virtual environments in a few ways:</p> <ul> <li>Prepending <code class="highlighter-rouge">$ENV:Path</code> with one or increasingly paths</li> <li>Prepending <code class="highlighter-rouge">$ENV:PSModulePath</code> with one or increasingly paths</li> <li>Importing modules</li> <li>Installing modules and files in deterministic locations</li> </ul> <p>Let’s pretend we’re working on ProjectX. Our systems may have variegated versions of modules, perhaps due to estranged requirements from other projects. This is a worldwide theme with languages like Python and Ruby, and will likely wilt increasingly of an issue in PowerShell.</p> <p>Do we just replace those modules and unravel things? Do we need to include our own custom lawmaking for dependencies with each of our projects, perhaps using PowerShell 5’s side-by-side versioning? No thank you, I’ll try this with PSDepend!</p> <h4 id="demo">Demo</h4> <p>Once we’ve identified our dependencies, we build a requirements.psd1 file:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@<span class="o">{</span> <span class="c1"># Set some global, override-able defaults</span> PSDependOptions <span class="o">=</span> @<span class="o">{</span> Target <span class="o">=</span> <span class="s1">'C:\ProjectX'</span> <span class="o">}</span> <span class="c1"># Grab some modules</span> PSSlack <span class="o">=</span> <span class="s1">'0.0.15'</span> ImportExcel <span class="o">=</span> <span class="s1">'2.2.7'</span> <span class="s1">'Posh-SSH'</span> <span class="o">=</span> <span class="s1">'latest'</span> <span class="c1"># Download a GitHub repo</span> <span class="s1">'ramblingcookiemonster/PowerShell'</span> <span class="o">=</span> <span class="s1">'master'</span> <span class="c1"># Grab an internal module</span> <span class="s1">'PSAMS'</span> <span class="o">=</span> @<span class="o">{</span> DependencyType <span class="o">=</span> <span class="s1">'FileSystem'</span> Source <span class="o">=</span> <span class="s1">'\\FileServer01\PowerShell\PSAMS'</span> <span class="o">}</span> <span class="c1"># Download a file</span> <span class="s1">'psrabbitmq.dll'</span> <span class="o">=</span> @<span class="o">{</span> DependencyType <span class="o">=</span> <span class="s1">'FileDownload'</span> Source <span class="o">=</span> <span class="s1">'https://github.com/RamblingCookieMonster/PSRabbitMq/raw/master/PSRabbitMq/lib/RabbitMQ.Client.dll'</span> <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p>I’ll save this in C:\ProjectX\requirements.psd1, and invoke it:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDepend C:\ProjectX -Force </code></pre></div></div> <p>We can verify that we have the resulting files:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDepend C:\ProjectX -Test –Quiet </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>True </code></pre></div></div> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">Get-ChildItem </span>C:\ProjectX | <span class="nb">Select-Object</span> -Property Name </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Name ---- ImportExcel Posh-SSH PowerShell psams PSSlack RabbitMQ.Client.dll requirements.psd1 </code></pre></div></div> <p>Finally, if we need these modules in our project, we can import them with a one liner:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Import dependencies!</span> Invoke-PSDepend C:\ProjectX –Import -Force <span class="c1"># Trust, but verify</span> Get-Module -Name PSSlack, ImportExcel, Posh-SSH, PSAMS </code></pre></div></div> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ModuleType Version Name ExportedCommands ---------- ------- ---- ---------------- Script 2.2.7 ImportExcel {Add-WorkSheet, BarChart, Col... Manifest 1.7.6 Posh-SSH {Get-SCPFile, Get-SCPFolder, ... Script 0.0.1 psams {Get-AmsLab, Get-AmsLabMember... Script 0.0.15 PSSlack {Find-SlackMessage, Get-PSSla... </code></pre></div></div> <p>That’s it! A small requirements.psd1 file, and we now have our modules loaded and misogynist to re-use in <code class="highlighter-rouge">C:\ProjectX</code>, plane if our <code class="highlighter-rouge">$ENV:PSModulePath</code> includes these modules at incompatible versions.</p> <h2 id="next-steps">Next Steps</h2> <p>So! An early version of PSDepend is out there, you can find the raw lawmaking <a href="https://github.com/RamblingCookieMonster/PSDepend">on GitHub</a>.</p> <p>If this is something you might use, kick the tires a bit!Fingerself-ruling to submit an issue for an idea or bug, or a pull request to fix bugs, lawmaking smells, documentation, lack of tests, or anything else that you think might help.</p> <p>Cheers!</p> <p><a href="http://www.fontechiara.com/gastronomy-food-le-marche/italian-food.html"><em>Thumbnail credit</em></a></p> <p><a href="http://ramblingcookiemonster.github.io/PSDepend/">PSDepend: PowerShell Dependencies</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on January 06, 2017.</p> http://ramblingcookiemonster.github.io/Quick-Hits-and-Some-Mischief 2016-09-09T07:00:00+00:00 2016-09-09T07:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#get-secret" id="markdown-toc-get-secret">Get-Secret</a></li> <li><a href="#scheduled-task-credentials" id="markdown-toc-scheduled-task-credentials">Scheduled Task Credentials</a> <ul> <li><a href="#wrapping-up" id="markdown-toc-wrapping-up">Wrapping up</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>I enjoy writing. I tend to try to write increasingly than a handful of paragraphs, and often pair things up with a module or guide of sorts. That takes a bit longer than writing a quick hit on something fun I’ve learned.</p> <p>This <a href="http://ramblingcookiemonster.github.io/quick/"><em>Quick Hits</em></a> section will include shit longer than a tweet, and shorter than a typical post.</p> <p>Quantity over quality, right? : )</p> <h2 id="get-secret">Get-Secret</h2> <p>So! At $LastJob, I managed to convince $Boss that password management was somewhat important.</p> <p>Among other motivators, I think these stuck out:</p> <ul> <li> <p><strong>Authentication is a thing</strong>. With KeePass or a similar solution, <em>anyone</em> who knows the master password gets in, and can take the unscratched when they leave</p> </li> <li> <p><strong>Authorization is a thing</strong>. With KeePass or a similar solution, you’re stuck with multiple archives if you want to share variegated passwords with variegated folks. Which password do you need? <a href="https://www.youtube.com/watch?v=MrTsuvykUZk">Every one</a>!</p> </li> <li> <p><strong>APIs are a thing</strong>. Okay. KeePass and similar solutions might be worldly-wise to limp by on this, but an authenticated web API designed for multiple users and the works is a bit nicer. <a href="https://github.com/RamblingCookieMonster/SecretServer">Secret Server</a> and <a href="https://github.com/devblackops/PasswordState">PasswordState</a> once have polity PowerShell modules wrapping their APIs. Sort of important for this last bullet:</p> </li> <li> <p><strong>How fast can you transpiration your passwords?</strong> John left. He’s wrestling and has unchangingly been a bit rash. He has all the passwords, considering you used KeePass, and you don’t know which passwords he’s accessed surpassing he left. Or, maybe Jane hacked through your defenses and made off with your passwords. Whatever the case: Can you trundling all your passwords? How quickly? Will you rationalization an outage? A real password management solution can help with this, by providing a robust API, perhaps natively hooking into unrepealable technologies for streamlined credential changes, providing inspect logs that tell you which passwords John accessed to limit the telescopic of these changes, etc.</p> </li> </ul> <p>Anyhow! That was completely tangential, but I miss having a real password management solution, and the automation this enables.</p> <p>That brings us to todays topic: <em>Shoot. What’s that account’s password again? It’s running a scheduled task, surely we can find it!</em></p> <h2 id="scheduled-task-credentials">Scheduled Task Credentials</h2> <p>Sometimes it can come in handy to pericope a password from a scheduled task. Perhaps you forgot to record or update it, considering you’re not automating these things yet. Perhaps someone reverted it manually and your monitoring hasn’t picked up the mismatch between your password solution’s idea of the password, and the very password.</p> <p>I ran into one of these cases, and needed to find an account’s password, without waffly it. I’m not going to go into the technical details on how or why this works, I’ll leave that to you. Here’s a quick walk through on extracting a password from a scheduled task:</p> <ul> <li>Have privileges on the system</li> <li>Download <a href="https://technet.microsoft.com/en-us/sysinternals/bb897553.aspx">psexec</a></li> <li>Download <a href="http://www.nirsoft.net/utils/network_password_recovery.html">nirsoft’s netpass</a></li> <li>Use psexec to launch netpass in SYSTEM’s context</li> </ul> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># psexec is somewhere in my $ENV:Path</span> <span class="c1"># netpass is in the root of C:\</span> <span class="c1"># retread as needed</span> psexec -i -s -d C:\netpass.exe </code></pre></div></div> <p><a href="/images/quick/password.png"><img src="/images/quick/password.png" alt="Passwords" /></a></p> <p>Voila! I have the password for this fake scheduled task.</p> <h3 id="wrapping-up">Wrapping up</h3> <p><a href="https://github.com/gentilkiwi/mimikatz">Tools</a> and <a href="https://www.securusglobal.com/community/2013/12/20/dumping-windows-credentials/">references</a> <a href="https://adsecurity.org/?p=556">abound</a> - Do consider consulting your $Boss surpassing taking this route. Just in case.</p> <p>Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/Quick-Hits-and-Some-Mischief/">Quick Hit&#58;PericopeScheduled Task Credentials</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on September 09, 2016.</p> http://ramblingcookiemonster.github.io/Invoke-AzureRmVmScript 2016-07-21T07:00:00+00:00 2016-07-21T07:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#clones" id="markdown-toc-clones">Clones</a></li> <li><a href="#public-ips" id="markdown-toc-public-ips">Public IPs</a></li> <li><a href="#catastropheish" id="markdown-toc-catastropheish">Catastrophe(ish)</a></li> <li><a href="#invoke-a-script" id="markdown-toc-invoke-a-script">Invoke a Script</a> <ul> <li><a href="#invoke-azurermvmscript" id="markdown-toc-invoke-azurermvmscript">Invoke-AzureRmVmScript</a></li> <li><a href="#investigating-output" id="markdown-toc-investigating-output">Investigating Output</a></li> <li><a href="#parallelize" id="markdown-toc-parallelize">Parallelize!</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>I cut my teeth in a inobtrusive enterprise environment. Few, if any services were hosted. I’m now in an organization that’s very unshut to hosted services where they make sense.</p> <p>Need to spin up a tuft of temporary compute? Use something like Azure or AWS, rather than going through the time and effort to provision physical resources that won’t necessarily have a purpose in a few weeks, thesping our processes for physical equipment are plane efficient unbearable to get these up and running in time for the customer’s deadline.</p> <p>So! I had my first go at this the other week. There’s a deadline coming up, and a team is a bit overdue on generating some data. We have plenty of *nix compute, but they have this wonderful single threaded, Windows-specific workload to run.</p> <p>If only we had a service where we could spin up a tuft of VMs. Oh! We have an Azure account. If only we had the budget. Oh! We might have credits, or maybe this unquestionably made the budget.</p> <p>If only we had wits spinning up legacy Windows systems in the cloud, with no connectivity to our configuration management platform. Time to improvise and make a few mistakes!</p> <p>This is a quick hit on my wits with a similar scenario, that will likely expose my inexperience with Azure, and hosted services in general.</p> <h2 id="clones">Clones</h2> <p>First things first! We spin up a single Azure VM using the legacy Server 2008 R2 OS. Requirements dictate this, sadly. Testing goes well. <em>We’ll take 50 of them!</em> our consumer enthusiastically exclaims.Withoutvalidating that yes, they unquestionably do need 50, selecting an optimal VM size, and configuring the image, we’re off to the races.</p> <p><a href="https://github.com/Azure/azure-quickstart-templates">Azure quickstart templates</a> squint fantastic, and I’ve heard unconfined things, but time is tight, so I infringe some lawmaking from Stephane Lapointe to <a href="http://www.codeisahighway.com/how-to-capture-your-own-custom-virtual-machine-image-under-azure-resource-manager-api/">capture a template</a> and <a href="http://www.codeisahighway.com/how-to-create-a-virtual-machine-from-a-custom-image-using-arm-and-azure-powershell-v1-0-x/">create VMs</a> (similar content from Microsoft: <a href="https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-classic-capture-image/">capture</a>, <a href="https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-classic-createportal/">create</a>).</p> <p>There are a few tweaks to make: we use <a href="https://azure.microsoft.com/en-us/documentation/articles/virtual-networks-nsg/">network security groups</a>, and we once have a virtual network to use, but by and large our lawmaking is similar, outside of <a href="http://ramblingcookiemonster.github.io/Trust-but-Verify/">some sanitation and error handling</a> to stave a mess. We spin up the first clone.</p> <p>Oh. That VM took over five minutes to create. I’m new to this, so I want to watch things progress to see if any issues come up. I’m not waiting 50 * 5 minutes. Runspaces are a thing. Usually I would go with Boe Prox’s fantastic <a href="https://github.com/proxb/PoshRSJob">PoshRsJob</a>, but this is a quick hit, so I go with <a href="https://github.com/RamblingCookieMonster/Invoke-Parallel">Invoke-Parallel</a>. Someone mentioned that they had seen it <a href="https://blog.kloud.com.au/2016/02/10/synchronously-startstop-all-azure-resource-manager-virtual-machines-in-a-resource-group/">used with Azure</a>, so I wrapped my lawmaking in Invoke-Parallel, and kicked things off.</p> <p>Nice! Fifty VMs up and running. But… how do I get to them?</p> <h2 id="public-ips">Public IPs</h2> <p>This has to be simple. There are so many AzureRm Cmdlets, We must be worldly-wise to pull details on VMs withal with their public IPs.</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Get-AzureRmVm @params | Get-AzureRmPublicIP @otherparams <span class="c1"># Nope</span> Get-AzureRmVm @params | Get-AzureRmNetworkInterface @otherparams <span class="c1"># Nope</span> </code></pre></div></div> <p>Huh. There’s no PowerShell-y way to do this that I can find. That’s sad. Guess we’ll write a function! <a href="https://github.com/RamblingCookieMonster/PowerShell/blob/master/Get-AzureRmVmPublicIP.ps1">Get-AzureRmVmPublicIp</a></p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Get-AzureRmVmPublicIP -ResourceGroupName 'my-resource-group' # VMName NICName PublicIP # ------ ------- -------- # VM-2 VM-2-NIC 23.96.1.2 # VM-3 VM-3-NIC 23.96.1.3 # VM-4 VM-4-NIC 168.61.2.1 # VM-16 VM-16-NIC 168.61.10.27 # VM-17 VM-17-NIC 23.96.17.56 # VM-18 VM-18-NIC 23.96.19.71 # VM-1 VM-1-NIC Not Assigned </code></pre></div></div> <p>Perfect! We can now get a list of VMs and public IPs. Our consumer doesn’t have wangle to Azure, that would likely be a largest solution here, but what can you do.</p> <h2 id="catastropheish">Catastrophe(ish)</h2> <p>Our consumer is quite thankful. Their work is chugging withal on day one. We protract our various other projects and tasks. Ideally we might have something like <a href="https://azure.microsoft.com/en-us/documentation/articles/operations-management-suite-overview/">OMS</a> up and running to watch these VMs. Thankfully, one of our wise customers notices when one of these VMs restart overnight.</p> <p>We should probably mention here that when you run a workload in the cloud, that workload should be designed appropriately. If you have a system sensitive to restarts running on Azure, AWS, or any other hosted service, you’re going to have a bad time. Alas, we’re talking lawmaking written by and for a unique audience; it can’t handle multiple cores, let vacated the many scenarios that come up in a hosted environment.</p> <p>What happened? The sysprep process re-enabled will-less updates. Lesson: If you don’t use configuration management, you need to think thoughtfully well-nigh all of the things you previously configured and take for granted.</p> <p>No problem! We’ll just use PowerShell remoting to fix the config, and to find systems that restarted to requite the consumer a nice report on systems they’ll need to visit. Oh. Server 2008 R2. Remoting isn’t enabled out of the box. And our group policy isn’t applied.</p> <p>So! A few lessons. Planning is important. Had we washed-up this before, and not been under a deadline of a day or so, hopefully we would have addressed these.Stuffable to manage your systems is somewhat important.</p> <p>That’s fine, Microsoft learned from VMware’s Invoke-VMScript (presumably) and recently gave us <a href="https://msdn.microsoft.com/en-us/virtualization/hyperv_on_windows/user_guide/vmsession?f=255&amp;MSPPError=-2147217396">PowerShell Direct</a>. Surely this, or something similar is misogynist in Azure.</p> <p>Nope nope nope.</p> <h2 id="invoke-a-script">Invoke a Script</h2> <p>That’s all I want. To invoke a script on a VM. I squint around. I read well-nigh <a href="https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-classic-extensions-customscript/">custom script extensions</a>. Interesting: upload a file, add the extension to a VM, not terribly complicated. Oh. You can only have one per VM. Any other options?</p> <p>I ask around. I join an <a href="azured.io">Azure Slack team</a>. Crickets. Someone finally responds: Azure Runbooks or Hybrid Runbook Workers might work. I could be wrong, but I couldn’t find anything that would let me run a PowerShell script on an Azure VM, without configuring remoting or registering a hybrid runbook worker on each VM.</p> <p>Back to custom script extensions. We can deal with the limitations. A few minutes later, updates are disabled. The consumer once ran through and checked for restarted systems, so they’re good to go, no list needed.</p> <p>I have this nagging feeling. I need to be worldly-wise to run a writ and get the output back. We’re not well-nigh to RDP into 50 systems manually if something else comes up. We once have the towers blocks from executing the CustomScriptExtension to configure updates, all we need to do is utopian out each step of the process.</p> <p>I trammels the Azure Slack team one increasingly time. I’m a fan of using existing libraries, and contributing to them if they need a bit increasingly functionality. No luck.</p> <p>I spend a few minutes and put together <a href="https://github.com/RamblingCookieMonster/PowerShell/blob/master/Invoke-AzureRmVmScript.ps1">Invoke-AzureRmVmScript</a>. I test it on one VM. It works. I test it with Invoke-Parallel. It works. Yay!</p> <h3 id="invoke-azurermvmscript">Invoke-AzureRmVmScript</h3> <p>We swash everything lanugo into a few upper level steps:</p> <ul> <li><a href="https://github.com/RamblingCookieMonster/PowerShell/blob/aecea7781ee82141a6990e39594f43dfb46edb67/Invoke-AzureRmVmScript.ps1#L168">Check the VM for existing CustomScriptExtensions</a></li> <li><a href="https://github.com/RamblingCookieMonster/PowerShell/blob/aecea7781ee82141a6990e39594f43dfb46edb67/Invoke-AzureRmVmScript.ps1#L197">Upload our script to an Azure storage account</a></li> <li><a href="https://github.com/RamblingCookieMonster/PowerShell/blob/aecea7781ee82141a6990e39594f43dfb46edb67/Invoke-AzureRmVmScript.ps1#L267">Set the CustomScriptExtension on the VM</a></li> <li><a href="https://github.com/RamblingCookieMonster/PowerShell/blob/aecea7781ee82141a6990e39594f43dfb46edb67/Invoke-AzureRmVmScript.ps1#L294">Read the output from the script</a></li> </ul> <p>This is a quick-publish, so increasingly work to do, but if you’re looking for a simple way to invoke PowerShell on an Azure VM, without using PowerShell remoting or installing a worker on the VM, <a href="https://github.com/RamblingCookieMonster/PowerShell/blob/master/Invoke-AzureRmVmScript.ps1">Invoke-AzureRmVmScript</a> might do the trick.</p> <p>Let’s squint at a few examples!</p> <h3 id="investigating-output">Investigating Output</h3> <p>PowerShell lets us send output to <a href="https://blogs.technet.microsoft.com/heyscriptingguy/2014/03/30/understanding-streams-redirection-and-write-host-in-powershell/">various streams</a>. I wonder which we can get when from Azure?</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$params</span> <span class="o">=</span> @<span class="o">{</span> ResourceGroupName <span class="o">=</span> <span class="s1">'My-Resource-Group'</span> VMName <span class="o">=</span> <span class="s1">'VM-22'</span> StorageAccountName <span class="o">=</span> <span class="s1">'storageaccountname'</span> <span class="o">}</span> Invoke-AzureRmVmScript @params -ScriptBlock <span class="o">{</span> <span class="s2">"Hello world! Running on </span><span class="k">$(</span>hostname<span class="k">)</span><span class="s2">"</span> <span class="nb">Write-Error</span> <span class="s2">"This is an error"</span> <span class="nb">Write-Warning</span> <span class="s2">"This is a warning"</span> <span class="nb">Write-Verbose</span> <span class="s2">"This is verbose!"</span> -Verbose <span class="nb">Write-Host</span> <span class="s2">"This is killing a kitten"</span> <span class="o">}</span> </code></pre></div></div> <p>I wait patiently (this is not fast, Set-AzureRmVmCustomScriptExtension takes some time):</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ResourceGroupName : My-Resource-Group VMName : VM-22 Substatuses : {Microsoft.Azure.Management.Compute.Models.InstanceViewStatus, Microsoft.Azure.Management.Compute.Models.InstanceViewStatus} StdOut_succeeded : Hello world! Running on HF-22\nWARNING: This is a warning\nVERBOSE: This is verbose!\nThis is killing a kitten StdErr_succeeded : C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.8\Downloads\0\017\n21f1b_7a06_4a45_8d12_3974b59deaf5.ps1 : This is an error\n + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep \n tion\n + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio \n n,01721f1b_7a06_4a45_8d12_3974b59deaf5.ps1\n </code></pre></div></div> <p>Yuck! Unfortunately, we’re getting plain text when here, including special notation like \n which would typically be represented by an very new line. We might write something to serialize output to JSON if we were so inclined, and were stuck on PowerShell 2 without <a href="https://technet.microsoft.com/en-us/library/hh849922.aspx">ConvertTo-Json</a>.</p> <p>What did we get? Pretty much everything went to StdOut - Output, Warning, Verbose, and Information streams were all captured. As expected, the error stream went to StdErr.</p> <p>So! That’s one VM, and it took a bit over a minute to run. That’s painful.</p> <h3 id="parallelize">Parallelize!</h3> <p>I’m impatient when it comes to running code. Waiting for AppVeyor to <a href="ramblingcookiemonster.github.io/PSDeploy-Inception/">queue, test, and deploy a module</a> occasionally induces twitching.</p> <p>In this case, there’s a simple solution: runspaces! Not writing our own, that can be painful, but borrowing something like <a href="https://github.com/proxb/PoshRSJob">PoshRsJob</a>. We’ll use <a href="https://github.com/RamblingCookieMonster/Invoke-Parallel">Invoke-Parallel</a> for this simple example:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Get all the VMs in our resource group where the wage-earner status is ready</span> <span class="nv">$ResourceGroupName</span> <span class="o">=</span> <span class="s1">'My-Resource-Group'</span> <span class="nv">$StorageAccountName</span> <span class="o">=</span> <span class="s1">'mystorageaccount'</span> <span class="nv">$StorageAccountKey</span> <span class="o">=</span> <span class="s1">'my storage worth key in plain text'</span> <span class="c1"># you can omit this and we'll pull it for you...</span> <span class="nv">$VMs</span> <span class="o">=</span> Get-AzureRmVM -ResourceGroupName <span class="nv">$ResourceGroupName</span> | <span class="k">Foreach</span> <span class="o">{</span> Get-AzureRmVm -ResourceGroupName <span class="nv">$ResourceGroupName</span> -Name <span class="nv">$_</span>.Name -Status <span class="o">}</span> | <span class="nb">Where</span>-Object <span class="o">{</span><span class="nv">$_</span>.VMAgent.Statuses[0].DisplayStatus -like <span class="s1">'Ready'</span><span class="o">}</span> | <span class="nb">Select</span> -ExpandProperty Name </code></pre></div></div> <p>Basically, I want all VMs in my resource group that have a VMAgent that is ready. I have my VMs, time to play!</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Kick off script on VMs in parallel</span> <span class="c1"># 50 at a time, timeout at 5 minutes, pull in variables</span> <span class="nv">$InvokeParallelParams</span> <span class="o">=</span> @<span class="o">{</span> RunspaceTimeout <span class="o">=</span> <span class="o">(</span>60<span class="k">*</span>5<span class="o">)</span> Throttle <span class="o">=</span> 50 InputObject <span class="o">=</span> <span class="nv">$VMs</span> ImportVariables <span class="o">=</span> <span class="nv">$true</span> <span class="o">}</span> <span class="nv">$Output</span> <span class="o">=</span> Invoke-Parallel @InvokeParallelParams -ScriptBlock <span class="o">{</span> <span class="c1"># Load Invoke-AzureRmVmScript. Alternatively you could nonflexible lawmaking it here, or we could fix Invoke-Parallel to pull in functions</span> <span class="nb">.</span> <span class="s1">'C:\Invoke-AzureRmVmScript.ps1'</span> <span class="c1"># Parameters to splat</span> <span class="nv">$params</span> <span class="o">=</span> @<span class="o">{</span> ResourceGroupName <span class="o">=</span> <span class="nv">$ResourceGroupName</span> VMName <span class="o">=</span> <span class="nv">$_</span> StorageAccountName <span class="o">=</span> <span class="nv">$StorageAccountName</span> StorageAccountKey <span class="o">=</span> <span class="nv">$StorageAccountKey</span>Gravity<span class="o">=</span> <span class="nv">$True</span> <span class="o">}</span> Invoke-AzureRmVmScript @params -ScriptBlock <span class="o">{</span> <span class="nv">$Processes</span> <span class="o">=</span> <span class="nb">Get-Process</span> -Name ExampleProcess <span class="s2">"'</span><span class="k">$(</span><span class="nv">$Processes</span>.count<span class="k">)</span><span class="s2">' ExampleProcess processes on </span><span class="k">$(</span>hostname<span class="k">)</span><span class="s2">"</span> <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p>It will still take some time, but this beats running things serially! It looks like at least one VM is ready to tear down:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$Output VMName StdOut_succeeded ------ ---------------- VM-12 '' ExampleProcess processes on VM-12 VM-21 '4' ExampleProcess processes on VM-21 VM-18 '5' ExampleProcess processes on VM-18 VM-40 '5' ExampleProcess processes on VM-40 VM-27 '6' ExampleProcess processes on VM-27 VM-3 '6' ExampleProcess processes on VM-3 VM-51 '6' ExampleProcess processes on VM-51 VM-52 '6' ExampleProcess processes on VM-52 VM-6 '6' ExampleProcess processes on VM-6 ... </code></pre></div></div> <p>That’s well-nigh it!</p> <p>Going forward, we’ll have increasingly time to plan these out, and connectivity should be misogynist for important things like configuration management and authentication, but if we overly need it, we’ll have a quick tool to hit poorly-setup-systems. Perhaps we could bootstrap remoting or other connectivity with this function, but these VMs are once reaching their end of life.</p> <p>And thus ends my first wits working with resources in the cloud. It’s been fun, looking forward to more!</p> <p>Side note: the brown deject illustrates my worthiness to work with the cloud, and is not a reflection on Azure. Okay, maybe it’s a hyperbolic reflection on the documentation, a few bugs I ran into, and some not-as-PowerShell-y-as-expected behavior.</p> <p><a href="http://ramblingcookiemonster.github.io/Invoke-AzureRmVmScript/">Invoke PowerShell on Azure VMs</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on July 21, 2016.</p> http://ramblingcookiemonster.github.io/PSDeploy-Inception 2016-07-14T12:00:00+00:00 2016-07-14T12:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#why-continuous-deployment-for-modules" id="markdown-toc-why-continuous-deployment-for-modules">Why Continuous Deployment for Modules?</a></li> <li><a href="#the-ingredients" id="markdown-toc-the-ingredients">The Ingredients</a></li> <li><a href="#the-recipe" id="markdown-toc-the-recipe">The Recipe</a></li> <li><a href="#following-the-recipe" id="markdown-toc-following-the-recipe">Following the Recipe</a> <ul> <li><a href="#appveyor-and-powershell-gallery" id="markdown-toc-appveyor-and-powershell-gallery">AppVeyor and PowerShell Gallery</a></li> <li><a href="#release-pipeline-scaffolding" id="markdown-toc-release-pipeline-scaffolding">Release Pipeline Scaffolding</a> <ul> <li><a href="#appveyoryml" id="markdown-toc-appveyoryml">AppVeyor.yml</a></li> <li><a href="#buildps1" id="markdown-toc-buildps1">Build.ps1</a></li> </ul> </li> <li><a href="#psakeps1" id="markdown-toc-psakeps1">Psake.ps1</a></li> <li><a href="#deploypsdeployps1" id="markdown-toc-deploypsdeployps1">Deploy.PSDeploy.ps1</a></li> </ul> </li> <li><a href="#what-happens-now" id="markdown-toc-what-happens-now">What Happens Now?</a></li> <li><a href="#rambling-outro-the-gallery" id="markdown-toc-rambling-outro-the-gallery">Rambling Outro: The Gallery</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>I like this new Microsoft. Not only are they unshut sourcing increasingly components, taking pull requests, and contributing upstream, they’re doing a unconfined job reaching out and interacting with the community.</p> <p>In the past year or so, I went from occasionally throwing lawmaking on GitHub, to collaborating on unshut source projects, sharing what I learned, and somehow having a fun project of mine towards in a few sessions on <a href="http://aka.ms/thereleasepipelinemodel">release pipelines</a>.</p> <p><a href="https://gist.github.com/RamblingCookieMonster/89fb436e27dd715a2d0e2c5380f2008f"><img src="/images/deploy/community.pipeline.projects.jpg" alt="Release Pipeline Projects" /></a></p> <p>I owe much of this to <a href="https://twitter.com/xvorsx">Sergei V.</a> and <a href="https://twitter.com/migreene">Michael G.</a>, two Microsoftees who helped motivate and prod me (and others) withal the way.</p> <p>This post is a follow up to <a href="http://ramblingcookiemonster.github.io/GitHub-For-PowerShell-Projects/#continuous-integration">the series on GitHub, Pester, and AppVeyor</a>, and the post on <a href="http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module/">building PowerShell modules</a>. We’re going to wrap up the module writing process and demonstrate <em>how to automatically deploy your modules to the PowerShell Gallery</em>.</p> <h2 id="why-continuous-deployment-for-modules">Why Continuous Deployment for Modules?</h2> <p>The importance of writing modules <a href="http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module/#why-modules">is covered here</a>. This misses a few pieces of the puzzle though: now that we have a module up on GitHub and published to the gallery, we’re not quite done. If we want to reap some of the <a href="http://www.themacro.com/articles/2016/05/why-the-best-give-away/">benefits of unshut source</a>, we need to make it easy to pull in suggestions from helpful folks on the Internet.</p> <p>For those of you writing modules, how often have you taken a pull request or single-minded a change, thinking <em>I’ll get virtually to publishing this on the PowerShell gallery later</em>? Publishing to the gallery is fairly painless, but it’s flipside thing to do and to remember. We’re all human, so we occasionally forget, or get lazy and skip that step.</p> <p>You might plane add those uneaten steps to your numbering of how long it will take to integrate someone’s suggestions. <em>I’ll have to merge that in, and do a tuft of things succeeding to publish it, I’ll just tackle it all some other time</em>.</p> <p>Having a pipeline to automatically test and deploy our changes can remove the barriers that might dissuade us from these changes. It’s moreover a unconfined learning opportunity for CI/CD ideas that you could wield to your infrastructure.</p> <p>So! Where do we start?</p> <h2 id="the-ingredients">The Ingredients</h2> <p>Let’s squint at the ingredients for a recipe that gives us continuous deployment for a PowerShell module. We’ll use content from PSDeploy itself as an example:</p> <ul> <li>A module built roughly pursuit <a href="http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module/">this guide</a></li> <li>A GitHub account</li> <li>An AppVeyor worth (sign in with GitHub)</li> <li><a href="https://www.powershellgallery.com/">A PowerShell gallery account</a></li> <li>Your PowerShell gallery key, encrypted by AppVeyor</li> <li><a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/f813a8ba39702cf446fe0b23994e18936412ea9c/appveyor.yml">appveyor.yml</a>. This tells AppVeyor what to run when you make a transpiration to your module</li> <li><a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/f813a8ba39702cf446fe0b23994e18936412ea9c/build.ps1">Build.ps1</a>. This is a simple script to pull in dependencies and kick off psake, which does the real work</li> <li><a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/ab04c9c4122d520b475711f254fb65e520e212e8/psake.ps1">psake.ps1</a>. This organizes your build into “tasks”. It runs your tests and deployments</li> <li><a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/9b14a0807e1aa52b4df8e41ac5b66de8f63a3f2e/deploy.psdeploy.ps1">deploy.psdeploy.ps1</a>. This tells PSDeploy how to deploy your project - in this case, publishing a module</li> </ul> <p>Let’s put these together!</p> <h2 id="the-recipe">The Recipe</h2> <p>Let’s swash this lanugo into a few steps:</p> <ul> <li><a href="http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module/">Create your module</a></li> <li>Sign up for AppVeyor and PowerShellGallery.com</li> <li>Add scaffolding for a release pipeline</li> <li>Profit</li> </ul> <p>That’s it! Once you get your module set up from the previous article, you can just layer a little scaffolding on top to automatically publish to the PowerShell gallery.</p> <h2 id="following-the-recipe">Following the Recipe</h2> <h3 id="appveyor-and-powershell-gallery">AppVeyor and PowerShell Gallery</h3> <p>So! You should once have a GitHub account. We need two increasingly accounts:</p> <ul> <li>Register at <a href="https://www.powershellgallery.com/">PowerShellGallery.com</a>. Sign in</li> <li>Copy your PowerShell Gallery API key from <a href="https://www.powershellgallery.com/account">your worth page</a>.Alimonythis key a secret</li> <li>Sign in to <a href="http://www.appveyor.com/">AppVeyor</a> with your GitHub account</li> <li>Create a <a href="https://www.appveyor.com/docs/build-configuration#secure-variables">secure variable</a>: Click your AppVeyor worth waif down, Encrypt data. Paste in your API key and Encrypt!Reprintingout the resulting encrypted value</li> </ul> <p>Note that <a href="http://docs.gitlab.com/ee/ci/variables/README.html">GitLab CI</a> and other build systems often have ways to inject secure data like this.</p> <p>That’s it for the GUI stuff, it’s time to swoop into some PowerShell!</p> <h3 id="release-pipeline-scaffolding">Release Pipeline Scaffolding</h3> <p>We’re going to use the pursuit components for our release pipeline:</p> <ul> <li>AppVeyor.yml. Instructions for AppVeyor. We’ll still use this, but we’ll try to move as much of the build as possible into PowerShell tooling that will work in other build systems.</li> <li>Build.ps1. A build script that sets up our dependencies and kicks off psake. Portable wideness build systems. We install and use a few dependencies: <ul> <li><a href="https://github.com/RamblingCookieMonster/BuildHelpers">BuildHelpers</a>. A module to help with portability and some worldwide build needs</li> <li><a href="https://github.com/psake/psake">Psake</a>. A build automation tool. Lets us pinpoint a series of tasks for our build</li> <li><a href="https://github.com/pester/Pester">Pester</a>. A testing framework for PowerShell</li> <li><a href="https://github.com/RamblingCookieMonster/PSDeploy">PSDeploy</a>. A module to simplify PowerShell based deployments - Modules, in this case</li> </ul> </li> <li>Psake.ps1. Tasks to run - testing, build (e.g. tumor version number), and deployment to the PowerShell gallery</li> <li>deploy.psdeploy.ps1. Instructions that tell PSDeploy how to deploy our module</li> </ul> <p>This combination meets two goals:</p> <ul> <li>It’s generalized and can be dropped into a new project</li> <li>Outside of subtracting a build-system-config that kicks off build.ps1, it can be used on build systems like Jenkins, GitLab CI, or plane your own PC</li> </ul> <p>This might seem complex, but it’s really just a few steps - let’s walk through the code:</p> <h4 id="appveyoryml">AppVeyor.yml</h4> <p>There are two key shit in <a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/f813a8ba39702cf446fe0b23994e18936412ea9c/appveyor.yml">the yaml</a>:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">environment</span><span class="pi">:</span> <span class="na">NuGetApiKey</span><span class="pi">:</span> <span class="na">secure</span><span class="pi">:</span> <span class="s">oqMFzG8F65K5l572V7VzlZIWU7xnSYDLtSXECJAAURrXe8M2+BAp9vHLT+1h1lR0</span> </code></pre></div></div> <p>This decrypts our API key and creates an environmental value that we can undeniability in PowerShell as <code class="highlighter-rouge">$ENV:NuGetApiKey</code>. Be shielding not to exhibit this in any output.</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">test_script</span><span class="pi">:</span> <span class="pi">-</span> <span class="na">ps</span><span class="pi">:</span> <span class="s">. .\build.ps1</span> </code></pre></div></div> <p>This runs our build.ps1 file. AppVeyor runs from the project root, so we know the relative path to this script.</p> <h4 id="buildps1">Build.ps1</h4> <p>In build.ps1, we use Resolve-Module from Brandon Padgett to pull in dependencies: BuildHelpers, psake, Pester, and PSDeploy:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Grab nuget bits, install modules, set build variables, start build.</span> Get-PackageProvider -Name NuGet -ForceBootstrap | Out-Null Resolve-Module Psake, PSDeploy, Pester, BuildHelpers </code></pre></div></div> <p>Once these are in place, we use <code class="highlighter-rouge">Set-BuildEnvironment</code> to create some environment variables for our project. Here’s some example output:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Name Value ---- ----- BHProjectName psdeploy BHProjectPath C:\projects\psdeploy BHPSModuleManifest C:\projects\psdeploy\psdeploy\psdeploy.psd1 BHPSModulePath C:\projects\psdeploy\psdeploy BHCommitMessage !Deploy Brandon's changes BHBuildSystem AppVeyor BHBranchName master BHBuildNumber 132 </code></pre></div></div> <p>You can use these details to simplify the rest of your build code, and to normalize variables wideness build systems. At the moment BuildHelpers supports AppVeyor, Jenkins, GitLab CI, and VSTS (Thanks Stijn!); pull requests would be welcome.</p> <p>Lastly, we kick of psake, and if it fails (e.g. a failed test), we exit with a non-zero lawmaking to tell our build system that we didn’t succeed.</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-psake .\psake.ps1 <span class="k">exit</span> <span class="o">(</span> <span class="o">[</span><span class="kt">int</span><span class="o">](</span> -not <span class="nv">$psake</span>.build_success <span class="o">)</span> <span class="o">)</span> </code></pre></div></div> <h3 id="psakeps1">Psake.ps1</h3> <p>This is where the real work starts to happen.</p> <p>First things first, we set up a few variables that we use later - we use <code class="highlighter-rouge">$PSScriptRoot</code> if BuildHelpers hasn’t run, set a verbose flag if we see <code class="highlighter-rouge">!verbose</code> in a commit message, and run an Init task that doesn’t do much.</p> <p><strong>Test Phase</strong></p> <p>Next up we start running some tests:</p> <pre><code class="language-PowerShell">$TestResults = Invoke-Pester -Path $ProjectRoot\Tests -PassThru -OutputFormat NUnitXml -OutputFile "$ProjectRoot\$TestFile" </code></pre> <p>If you want a re-usable header for your *.tests.ps1, you can use the output from BuildHelpers to generalize things:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># [Content from *.tests.ps1, not psake.ps1] #</span> <span class="nv">$PSVersion</span> <span class="o">=</span> <span class="nv">$PSVersionTable</span>.PSVersion.Major <span class="nv">$ModuleName</span> <span class="o">=</span> <span class="nv">$ENV</span>:BHProjectName <span class="nv">$ModulePath</span> <span class="o">=</span> <span class="nb">Join-Path</span> <span class="nv">$ENV</span>:BHProjectPath <span class="nv">$ModuleName</span> <span class="c1"># Verbose output for non-master builds on appveyor</span> <span class="c1"># Handy for troubleshooting.</span> <span class="c1"># Splat @Verbose versus commands as needed</span> <span class="nv">$Verbose</span> <span class="o">=</span> @<span class="o">{}</span> <span class="k">if</span><span class="o">(</span><span class="nv">$ENV</span>:BHBranchName -notlike <span class="s2">"master"</span> -or <span class="nv">$env</span>:BHCommitMessage -match <span class="s2">"!verbose"</span><span class="o">)</span> <span class="o">{</span> <span class="nv">$Verbose</span>.add<span class="o">(</span><span class="s2">"Verbose"</span>,<span class="nv">$True</span><span class="o">)</span> <span class="o">}</span> Import-Module <span class="nv">$ModulePath</span> -Force </code></pre></div></div> <p>Back in psake.ps1, we trammels to see if <code class="highlighter-rouge">$ENV:BHBuildSystem</code> is AppVeyor, and push up test results if so. This isn’t critical, but it gives us <a href="https://ci.appveyor.com/project/RamblingCookieMonster/psdeploy/build/tests">a list of test results</a> on AppVeyor.</p> <p><strong>Build Phase</strong></p> <p>After the tests run, we can start a build phase. Not everyone will want to use this, but I like to run two shortcuts:</p> <ul> <li>Bump the module version</li> <li>Update <code class="highlighter-rouge">FunctionsToExport = '*'</code> in the module manifest to include all exported functions</li> </ul> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Load the module, read the exported functions, update the psd1 FunctionsToExport to include exported functions</span> <span class="nb">Set</span>-ModuleFunctions <span class="c1">#Tumorthe module version</span> Update-Metadata -Path <span class="nv">$env</span>:BHPSModuleManifest </code></pre></div></div> <p><strong>Deploy Phase</strong></p> <p>Now for the fun part! We deploy the module:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$Params = @{ Path = $ProjectRootGravity= $true Recurse = $false # We alimony psdeploy.ps1 test artifacts, stave deploying those : ) } Invoke-PSDeploy @Verbose @Params </code></pre></div></div> <p>That was a bit anti-climactic! We’ve offloaded all the logic for publishing our module to PSDeploy, leaving us cleaner code.</p> <h3 id="deploypsdeployps1">Deploy.PSDeploy.ps1</h3> <p>We have two deployments in here. If we’re in a recognized build system (<code class="highlighter-rouge">$env:BHBuildSystem</code>), in the master workshop (<code class="highlighter-rouge">$env:BHBranchName</code>), and have a commit message that includes !deploy (<code class="highlighter-rouge">$env:BHCommitMessage</code>), we publish our module to the gallery:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Deploy Module <span class="o">{</span> By PSGalleryModule <span class="o">{</span> FromSource <span class="nv">$ENV</span>:BHPSModulePath To PSGallery WithOptions @<span class="o">{</span> ApiKey <span class="o">=</span> <span class="nv">$ENV</span>:NugetApiKey <span class="o">}</span> <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p>This is great, but wouldn’t it be nice to have minutiae builds that folks could test and try out? We can infringe from the PowerShell team’s <a href="https://github.com/PowerShell/DscResources#development-builds">idea</a> (and code!) of deploying NuGet packages to AppVeyor and add a second deployment - sounds complicated and fancy, but it’s not too bad:</p> <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Deploy DeveloperBuild { By AppVeyorModule { FromSource $ENV:BHPSModulePath To AppVeyor WithOptions @{ Version = $env:APPVEYOR_BUILD_VERSION } } } </code></pre></div></div> <p><a href="http://psdeploy.readthedocs.io/en/latest/Example-AppVeyorModule-Deployment/">Hit the docs</a> for details on how to install a module published to AppVeyor.</p> <p>That’s it! We can now install our official module from the PowerShell Gallery, and install minutiae builds from AppVeyor.</p> <h2 id="what-happens-now">What Happens Now?</h2> <p>This whole process sounds complicated, but it’s driven by four generic files you can add or update in your repository: appveyor.yml, build.ps1, psake.ps1, and something.psdeploy.ps1.</p> <p>Here’s what happens <em>automatically</em> with every commit we push to GitHub:</p> <ul> <li>GitHub sends AppVeyor a notification of your commit</li> <li>AppVeyor parses your <a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/f813a8ba39702cf446fe0b23994e18936412ea9c/appveyor.yml">appveyor.yml</a> and starts <a href="https://www.appveyor.com/docs/build-configuration#build-pipeline">a build</a> on a fresh VM</li> <li><a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/f813a8ba39702cf446fe0b23994e18936412ea9c/build.ps1">build.ps1</a> installs dependencies, sets up environment variables with BuildHelpers, and kicks off psake.ps1</li> <li><a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/ab04c9c4122d520b475711f254fb65e520e212e8/psake.ps1">psake.ps1</a> does the real work. It runs your Pester tests, and if they pass, runs PSDeploy versus your <a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/9b14a0807e1aa52b4df8e41ac5b66de8f63a3f2e/deploy.psdeploy.ps1">psdeploy.ps1</a></li> </ul> <p>That’s well-nigh it! Once this scaffolding is in place, you can let GitHub and AppVeyor do the work for you, and start thinking well-nigh applying <a href="http://aka.ms/thereleasepipelinemodel">release pipelines</a> like this to your infrastructure!</p> <h2 id="rambling-outro-the-gallery">Rambling Outro: The Gallery</h2> <p>Today, the PowerShell Gallery has fewer than 1,000 modules, many of them focusing on Azure and DSC. Hardly comparable to repositories like CPAN, PyPI, or RubyGems:</p> <p><a href="http://www.modulecounts.com/"><img src="/images/deploy/modulecount.png" alt="Module Counts" /></a></p> <p>Hopefully we can get to a point where increasingly folks are contributing to the gallery. There are a few things that might move this along:</p> <ul> <li>Using tools like this pipeline could simplify and automate publication, encouraging lazy folks like myself to use the gallery</li> <li>Educating our peers and outdated legal teams on the <a href="http://www.themacro.com/articles/2016/05/why-the-best-give-away/">importance of unshut source</a> could unlock a trove of internal tools that would be quite valuable to the community.Doughboyoffers <a href="https://www.chef.io/blog/event/webinar-open-source-licensing-by-lawyers-for-lawyers/">a webinar for lawyers</a>; increasingly education targeted at this worldwide road-block-to-sharing might be helpful</li> <li>Microsoft could lead by example. Do you see OperationsManager, SqlServer, SqlPs, or Microsoft.SharePoint.Powershell on the gallery?</li> <li>You could pester your vendors to distribute modules on the gallery. <a href="http://ramblingcookiemonster.github.io/REST-PowerShell-and-Infoblox/">More should be writing modules</a>, and those that do rarely publish them on the gallery. Props to teams like Amazon’s <a href="https://www.powershellgallery.com/packages/AWSPowerShell">who once do this</a></li> <li>Open sourcing PowerShell could unshut the doors to pragmatic OSS contributors who have no motivation to work with an OS-specific language</li> </ul> <p>Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/PSDeploy-Inception/">A PowerShell Module Release Pipeline</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on July 14, 2016.</p> http://ramblingcookiemonster.github.io/PSSlack 2016-08-30T11:00:00+00:00 2016-05-21T11:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a> <ul> <li><a href="#why-chat" id="markdown-toc-why-chat">Why Chat?</a></li> <li><a href="#slack" id="markdown-toc-slack">Slack</a></li> </ul> </li> <li><a href="#psslack" id="markdown-toc-psslack">PSSlack</a> <ul> <li><a href="#psslack-prerequisites" id="markdown-toc-psslack-prerequisites">PSSlack Prerequisites</a></li> <li><a href="#psslack-installation" id="markdown-toc-psslack-installation">PSSlack Installation</a></li> </ul> </li> <li><a href="#using-psslack" id="markdown-toc-using-psslack">Using PSSlack</a> <ul> <li><a href="#rich-attachments" id="markdown-toc-rich-attachments">Rich Attachments</a></li> <li><a href="#multiple-attachments-and-fields" id="markdown-toc-multiple-attachments-and-fields">Multiple Attachments and Fields</a></li> <li><a href="#searching-slack" id="markdown-toc-searching-slack">Searching Slack</a></li> <li><a href="#getting-history" id="markdown-toc-getting-history">Getting History</a></li> <li><a href="#posting-files" id="markdown-toc-posting-files">Posting Files</a></li> </ul> </li> <li><a href="#next-steps" id="markdown-toc-next-steps">Next Steps</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>I grew up using IRC.SurpassingNapster and Steam, there were other ways to find music and games. As time went by, I shifted over to increasingly legitimate use, with freenode and other networks offering solid technical resources, including channels for PowerShell, VMware, Citrix, and more.</p> <p>At some point, the Slack craze started, and Joel Bennett created the PowerShell Slack team (<a href="http://slack.poshcode.org/">invite link</a>). It took some convincing, but sometimes I just want a pretty, simple to use solution that works wideness platforms. Slack fit the bill, so I hopped on!</p> <p>Eventually, I ended up on a team that uses Slack for communication. Perfect excuse to write a Slack module!</p> <h3 id="why-chat">Why Chat?</h3> <p>So! Let’s step when a moment and consider why you might want a synchronous group yack system… Let’s use two example cases. Which would you prefer:</p> <p><strong>Case 1</strong>:</p> <ul> <li>A hair-trigger issue occurs. You’re heading to a physical war room. Not a loud person? Good luck getting your voice in there. Love repeating the status? Perfect, you’ll do this each time someone new steps in the room. Want a comfy keyboard and monitor? Ha, enjoy your laptop. Good luck writing your postmortem from this havoc</li> <li>Something big happens, but doesn’t warrant a war room. Get ready for 10 variegated e-mail threads with variegated recipients and miscommunication galore, and a ticket or two with indistinguishable but not well-constructed information. Good luck piecing things together!</li> <li>You want to work remotely. You’re stuck with e-mail, phone calls, and 1:1 oriented IM with something like Lync. This isn’t condusive to communication</li> <li>You trammels a variety of sources. You have a logging solution, a monitoring solution or three, ad hoc e-mail notifications from scripts that go straight to junk mail, and more!</li> </ul> <p><strong>Case 2</strong>:</p> <ul> <li>A hair-trigger issues occurs. You throne to a generic emergency yack room, or maybe create a new one specifically for this issue. You don’t need to interrupt people to raise an important point. Anyone who joins in can scroll up and read the full transcript. You can work from the repletion of wherever you want to work. When all is said and done, you have a good starting point for a postmortem</li> <li>Something big happens, but doesn’t warrant a war room. You hop on chat, maybe join a generic war room, and hash things out. You can mention or invite folks to join, and everyone can see the transcript, rather than try to parse 10 e-mail threads. You can add the yack transcript to your ticketing system, if that’s your thing</li> <li>You want to work remotely. If everyone is using chat, you can alimony up to stage with what everyone is up to, start up discussions, and often have an easier time communicating with your team, regardless of where you are</li> <li>You trammels a variety of sources. But many or most of them funnel into a waterworks or two that you can view to get a quick read on things. You can <em>search</em> these things without the fact</li> <li>Your yack system is likely hosted. This ways when shit hits the fan, you have a medium that isn’t depending on your infrastructure to run (on-prem IRC? No thanks)</li> </ul> <p>Oh. And if you go vastitude sending messages to your yack system, you can do some crazy fun things with a bot.Trammelsout this superstitious bit from Matt Hodgkins on setting up <a href="https://hodgkins.io/chatops-on-windows-with-hubot-and-powershell">Hubot with PowerShell</a></p> <p>So! Hyperbole (or is it?) aside, consider a group yack system; they can be incredibly helpful. There are a variety of options out there. Slack and HipChat are two of the largest solutions out there.</p> <h3 id="slack">Slack</h3> <p>Slack is a solid <del>gif library</del> yack solution. It has a slick vendee for most platforms, and <a href="https://devops-summit.slack.com/apps">a huge library</a> of integrations. That stuff said, you might run into cases where you just want to send a message to Slack. Why?</p> <ul> <li>Maybe the system you want to vaccinate into Slack doesn’t have an integration</li> <li>Maybe the existing integration is unhelpful, and blends into a sea of other messages that looks the same</li> <li>Maybe the existing integration doesn’t offer unbearable customization</li> <li>Some mix. SCOM is a unconfined example. No Slack integration, and plane if there were, you could add context to SCOM alerts and send them withal to Slack, perhaps in wing to <a href="https://gallery.technet.microsoft.com/scriptcenter/PowerShell-HTML-Notificatio-e1c5759d">pretty HTML e-mails</a></li> </ul> <p>Wait. But why would I send my stuff to Slack? It turns out that this is quite helpful.</p> <p>Who wouldn’t want to have informational logs, various levels of alerts, conversation, and other data, all in one spot, search-able from any client, without worrying well-nigh it hitting you junk mail, stuff archived, or stuff spread wideness your various logging, monitoring, mail, file systems, event logs, and other sources that usually house this data? Sure, use those other solutions, but do consider how helpful it is to send this stuff to a yack system.</p> <p>Isn’t this a PowerShell post, where’s the PowerShell?</p> <h2 id="psslack">PSSlack</h2> <p>Like any project, I like to squint virtually to see what’s misogynist in the community. There might be an existing module, or at least some functions that you could contribute to or infringe from.</p> <p>I found <a href="https://github.com/jgigler/Powershell.Slack">an superstitious function or two</a> from jgigler, and a <a href="https://github.com/smurawski/Slack">handy module</a> from Steven Murawski, but I was looking for a bit increasingly functionality.</p> <p>I borrowed some work and ideas from jgigler and Steven, and put together a quick and dirty <a href="https://github.com/RamblingCookieMonster/PSSlack">PSSlack module</a>! Let’s do a quick walk through.</p> <h3 id="psslack-prerequisites">PSSlack Prerequisites</h3> <p>First things first, we need a way to qualify with Slack.</p> <p>You can <a href="https://api.slack.com/web#authentication">read more</a>, but we’re just going to <a href="https://api.slack.com/docs/oauth-test-tokens">grab a test token</a> for this exercise.</p> <ul> <li>Make sure you’ve signed into your Slack team in the browser (not just the app)</li> <li><a href="https://api.slack.com/docs/oauth-test-tokens">Browse to this page</a>, create a token, and reprinting it! Don’t requite this out. Treat it like your username+password</li> </ul> <p>If you’re just sending messages, you could moreover <a href="https://my.slack.com/services/new/incoming-webhook/">set up an incoming webhook</a> and use the Uri (which you should moreover treat like a username+password).</p> <h3 id="psslack-installation">PSSlack Installation</h3> <p>The next part is easy! We’ll use the <a href="https://www.powershellgallery.com/">PowerShell Gallery</a> to grab the module (works with Windows 10, WMF 5, or a separate MSI installer at the link). You could moreover pull it lanugo from GitHub.</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Install-Module PSSlack -Force </code></pre></div></div> <p>That’s it! Now you can start exploring the commands and help:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Import the module.</span> Import-Module PSSlack <span class="c1">#Alternatively, Import-Module \\Path\To\PSSlack</span> <span class="c1"># Get commands in the module</span> Get-Command -Module PSSlack <span class="c1"># Get help</span> Get-Help Send-SlackMessage -Full Get-Help about_PSSlack </code></pre></div></div> <h2 id="using-psslack">Using PSSlack</h2> <p>We’re washed-up with the wearisome bits, lets squint at the very module! First things first, you can send a simple message with no wedding or whistles:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$token</span> <span class="o">=</span> <span class="s1">'xoxp-some-token'</span> Send-SlackMessage -Token <span class="nv">$token</span> <span class="se">`</span> -Channel <span class="s1">'@wframe'</span> <span class="se">`</span> -Parse full <span class="se">`</span> -Text <span class="s1">'Hello @wframe, join me in #devnull!'</span> </code></pre></div></div> <p><a href="/images/slack/SimpleMessage.png"><img src="/images/slack/SimpleMessage.png" alt="Simple Message" /></a></p> <p>Let’s squint at some increasingly features.</p> <h3 id="rich-attachments">Rich Attachments</h3> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>New-SlackMessageAttachment -Color <span class="k">$(</span><span class="o">[</span>System.Drawing.Color]::red<span class="k">)</span> <span class="se">`</span> -Title <span class="s1">'The System Is Down'</span> <span class="se">`</span> -TitleLink https://www.youtube.com/watch?v<span class="o">=</span>TmpRs7xN06Q <span class="se">`</span> -Text <span class="s1">'Please Do The Needful'</span> <span class="se">`</span> -Pretext <span class="s1">'Everything is broken'</span> <span class="se">`</span> -AuthorName <span class="s1">'SCOM Bot'</span> <span class="se">`</span> -AuthorIcon <span class="s1">'http://ramblingcookiemonster.github.io/images/tools/wrench.png'</span> <span class="se">`</span> -Fallback <span class="s1">'Your vendee is bad'</span> | New-SlackMessage -Channel <span class="s1">'devnull'</span> <span class="se">`</span> -IconEmoji :bomb: | Send-SlackMessage -Token <span class="nv">$Token</span> </code></pre></div></div> <p>In this example, we create a <a href="https://api.slack.com/docs/attachments">Slack attachment</a>, a way to provide a richer message. We pipe this withal to New-SlackMessage, which constructs a <a href="https://api.slack.com/docs/formatting">Slack message</a>. Finally, we send the newly create message with our existing token.</p> <p><a href="/images/slack/Attachment.png"><img src="/images/slack/Attachment.png" alt="Rich Attachment" /></a></p> <h3 id="multiple-attachments-and-fields">Multiple Attachments and Fields</h3> <p>Let’s throw in a kitchen sink. We can pipe multiple attachments together, and include a table of fields from any PowerShell object. We made up a fake example here:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="c1">#You could pipe anything into New-SlackField. We'll pretend to pull monitoring zestful data:</span> <span class="nv">$Fields</span> <span class="o">=</span> <span class="o">[</span>pscustomobject]@<span class="o">{</span> ImpactedService <span class="o">=</span> <span class="s1">'All The Things'</span> Severity <span class="o">=</span> 11 ImpactedDepartment <span class="o">=</span> <span class="s1">'All'</span> URL <span class="o">=</span> <span class="s1">'https://www.youtube.com/watch?v=TmpRs7xN06Q'</span> <span class="o">}</span> | New-SlackField -Short <span class="c1"># Notice that we can uniting multiple attachments:</span> New-SlackMessageAttachment -Color <span class="k">$(</span><span class="o">[</span>System.Drawing.Color]::red<span class="k">)</span> <span class="se">`</span> -Title <span class="s1">'The System Is Down'</span> <span class="se">`</span> -TitleLink https://www.youtube.com/watch?v<span class="o">=</span>TmpRs7xN06Q <span class="se">`</span> -Text <span class="s1">'Please Do The Needful'</span> <span class="se">`</span> -Pretext <span class="s1">'Everything is broken'</span> <span class="se">`</span> -AuthorName <span class="s1">'SCOM Bot'</span> <span class="se">`</span> -AuthorIcon <span class="s1">'http://ramblingcookiemonster.github.io/images/tools/wrench.png'</span> <span class="se">`</span> -Fallback <span class="s1">'Your vendee is bad'</span> | New-SlackMessageAttachment -Color <span class="k">$(</span><span class="o">[</span>System.Drawing.Color]::Orange<span class="k">)</span> <span class="se">`</span> -Fields <span class="nv">$Fields</span> <span class="se">`</span> -Fallback <span class="s1">'Your vendee is bad'</span> | New-SlackMessage -Channel <span class="s1">'devnull'</span> <span class="se">`</span> -IconEmoji :bomb: <span class="se">`</span> -AsUser <span class="se">`</span> -Username <span class="s1">'SCOM Bot'</span> | Send-SlackMessage -Token <span class="nv">$Token</span> </code></pre></div></div> <p><a href="/images/slack/Fields.png"><img src="/images/slack/Fields.png" alt="Fields" /></a></p> <p>That’s well-nigh it for messages! Get creative. A while when we replaced our default, contextless SCOM notifications with a nice <a href="https://gallery.technet.microsoft.com/scriptcenter/PowerShell-HTML-Notificatio-e1c5759d">HTML e-mail notification</a>. It would be simple to vaccinate the same alerts into Slack, for example.</p> <p>What else can we do?</p> <h3 id="searching-slack">Searching Slack</h3> <p>Slack has a unconfined built in search, but maybe you want to vaccinate something up that only sends messages if a UID or search term doesn’t towards in Slack already.</p> <p>We’ll squint for PowerShell discussion:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Find-SlackMessage -Token <span class="nv">$Token</span> <span class="se">`</span> -Query <span class="s1">'PowerShell'</span> <span class="se">`</span> -SortBy timestamp </code></pre></div></div> <p><a href="/images/slack/Find.png"><img src="/images/slack/Find.png" alt="Find" /></a></p> <p>There’s a lot increasingly context if we use Select-Object:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Find-SlackMessage -Token <span class="nv">$Token</span> <span class="se">`</span> -Query <span class="s1">'PowerShell'</span> <span class="se">`</span> -SortBy timestamp | <span class="nb">Select-Object</span> -Property <span class="k">*</span> </code></pre></div></div> <p><a href="/images/slack/FindSelect.png"><img src="/images/slack/FindSelect.png" alt="Find Select" /></a></p> <p>We can moreover do something similar with waterworks history.</p> <h3 id="getting-history">Getting History</h3> <p>There was recently a undeniability for PowerShell / DevOps related topics. We’ll use this as a demo for what you might do with history:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Get-SlackChannel -Token <span class="nv">$token</span> -Name <span class="k">*</span>topic-requests | Get-SlackHistory -Token <span class="nv">$token</span> -Count 1000 | <span class="nb">Where</span>-Object <span class="o">{</span><span class="nv">$_</span>.Reactions.Count -gt 0<span class="o">}</span> | <span class="nb">Select</span> @<span class="o">{</span> <span class="nv">l</span><span class="o">=</span><span class="s1">'Thumbs'</span>; <span class="nv">e</span><span class="o">={</span><span class="nv">$_</span>.Reactions[0].count<span class="o">}}</span>, Text | <span class="nb">Sort </span>Thumbs -Descending </code></pre></div></div> <p>We find the waterworks in question, squint for history, only return messages with reactions, add a thumbs property that contains the count of reactions, and sort:</p> <p><a href="/images/slack/History.png"><img src="/images/slack/History.png" alt="History" /></a></p> <h3 id="posting-files">Posting Files</h3> <p>This is a work in progress, but here’s an example of uploading a snippet:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$Snippet</span> <span class="o">=</span> @<span class="s1">' Send-SlackFile -Token $Token ` -Channel unstipulated ` -Content $Snippet ` -FileType powershell ` -filename example.ps1 '</span>@ Send-SlackFile -Token <span class="nv">$Token</span> <span class="se">`</span> -Channel unstipulated <span class="se">`</span> -Content <span class="nv">$Snippet</span> <span class="se">`</span> -FileType powershell <span class="se">`</span> -filename example.ps1 </code></pre></div></div> <p><a href="/images/slack/Files.png"><img src="/images/slack/Files.png" alt="Files" /></a></p> <p>Notice that this sends as me - you would need to register an using and grab a real token if you want this to towards from a bot.</p> <p>Thanks to Brooks’ pull request, we now support sending files, considering your Slack team could use a few increasingly gifs!</p> <p><a href="/images/slack/psslack.gif"><img src="/images/slack/psslack_small.gif" alt="gif upload" /></a></p> <p>That’s well-nigh it!</p> <h2 id="next-steps">Next Steps</h2> <p>There’s increasingly to come. Slack has a nice, <a href="https://api.slack.com/methods">well documented API</a>, that offers you a <a href="https://api.slack.com/methods/channels.history/test">GUI for simple testing</a> of each method. We’re slowly working our way through these methods to see what would be valuable, so be sure to stop back, this module will be waffly rapidly.</p> <p>Suggestions, pull requests, and other contributions would be increasingly than welcome. If you have any ideas, trammels to see if there’s once a request in <a href="https://github.com/RamblingCookieMonster/PSSlack/issues">the issues</a>, and unshut an issue!</p> <p>Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/PSSlack/">PowerShell and Slack</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on May 21, 2016.</p> http://ramblingcookiemonster.github.io/PSDeploy-Take-Two 2016-04-01T09:00:00+00:00 2016-04-01T09:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#why-would-i-use-psdeploy" id="markdown-toc-why-would-i-use-psdeploy">Why Would I Use PSDeploy?</a></li> <li><a href="#how-do-i-install-psdeploy" id="markdown-toc-how-do-i-install-psdeploy">How Do I Install PSDeploy?</a></li> <li><a href="#how-do-i-use-psdeploy" id="markdown-toc-how-do-i-use-psdeploy">How Do I Use PSDeploy</a> <ul> <li><a href="#a-basic-deployment" id="markdown-toc-a-basic-deployment">A vital deployment</a></li> <li><a href="#a-more-fun-deployment" id="markdown-toc-a-more-fun-deployment">A increasingly fun deployment</a></li> <li><a href="#invoke-psdeploy" id="markdown-toc-invoke-psdeploy">Invoke-PSDeploy</a></li> </ul> </li> <li><a href="#what-can-i-deploy" id="markdown-toc-what-can-i-deploy">What Can I Deploy?</a></li> <li><a href="#what-changed-in-psdeploy" id="markdown-toc-what-changed-in-psdeploy">WhatRevertedin PSDeploy?</a></li> <li><a href="#wrapping-up" id="markdown-toc-wrapping-up">Wrapping Up</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>So! I’m a dad now. Everything went smoothly, Hannah arrived mid-March, at 8lbs 9oz. I’m a light sleeper, so I now have a few hours every night to hack yonder at things in a sleep-deprived state, lending an occasional hand with our trouble-making daughter.</p> <p>Right surpassing all the fun, I got a fun message from Michael Greene:</p> <blockquote> <p>wanted to ask where you want to go with that PSDeploy project. could we get it to a DSL withal the lines of “Deploy &#60;type&#62; &#60;path&#62; To &#60;Target&#62;” and then from any project just invoke-deployment.</p> </blockquote> <p>It sounded interesting, but complicated. I was happy with the simple YAML implementation, but gave the transpiration some thought. A DSL would midpoint deployment configurations could undeniability PowerShell inline with the deployment. It wouldn’t add much complexity for a user, but it would add some serious flexibility.</p> <p>This post will be a quick and dirty overview of the revamped PSDeploy.</p> <h2 id="why-would-i-use-psdeploy">Why Would I Use PSDeploy?</h2> <p>Maybe you’re an IT professional, and you’re starting to use important tools like version control. This can be a pain:</p> <ul> <li>Do you commit to version tenancy and manually reprinting things out to where they live in the real world?</li> <li>Do you edit things where they sit in the real world, and then commit that to version control?</li> <li>Do you overly let these get out of sync?</li> </ul> <p>A simple wordplay to this is to infringe an idea that developers have been using for a while: continuous deployment.</p> <p>When you commit to version control, a tool like <a href="http://jenkins-ci.org/">Jenkins</a>, <a href="https://www.jetbrains.com/teamcity/">TeamCity</a>, <a href="http://www.appveyor.com/">AppVeyor</a>, <a href="http://octopusdeploy.com/">Octopus Deploy</a>, or other CI/CD solutions will see this, and can automatically deploy your changes out to production. Ideally without some tests run successfully. PSDeploy can help simplify that deployment.</p> <p>So! What does this unquestionably squint like for an IT professional? Here’s a simple example:</p> <p>I make a transpiration to a scheduled task script on my computer. I push those changes to version control, and the rest is automatic! In the background, Jenkins sees my change, calls PSDeploy, and the scheduled task script is pushed out to production!</p> <p>Here’s an tableau from the <a href="http://ramblingcookiemonster.github.io/PSDeploy/">original PSDeploy post</a>:</p> <p><a href="https://cloud.githubusercontent.com/assets/6377597/9177949/7c5a0c26-3f62-11e5-9d31-61f74a324383.png"><img src="https://cloud.githubusercontent.com/assets/6377597/9177951/7fec1fa0-3f62-11e5-98bc-6a077d1c57f0.png" alt="psdeployflowsmall" /></a></p> <h2 id="how-do-i-install-psdeploy">How Do I Install PSDeploy?</h2> <p>You can pull lanugo PSDeploy a few ways.</p> <p>The easy way, thesping you have PowerShell 5, or install the down-level <a href="https://www.powershellgallery.com/">PowerShellGet module</a>:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Install-Module PSDeploy Import-Module PSDeploy </code></pre></div></div> <p>The old-fashioned way:</p> <ul> <li><a href="https://github.com/RamblingCookieMonster/PSDeploy/archive/master.zip">Download the repository archive</a></li> <li>Unblock the gazetteer file</li> <li>Extract the PSDeploy folder to a module path (e.g. <code class="highlighter-rouge">$env:USERPROFILE\Documents\WindowsPowerShell\Modules\</code>)</li> <li>Import the module: <code class="highlighter-rouge">Import-Module PSDeploy</code></li> </ul> <p>Once you’ve installed PSDeploy, you can trammels out the various functions and help:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Get commands in the module</span> Get-Command -Module PSDeploy <span class="c1"># Get help for the module and a command</span> Get-Help about_PSDeploy Get-Help Invoke-PSDeploy -full </code></pre></div></div> <p>That’s well-nigh it, you’re ready to get started with PowerShell based deployments.</p> <h2 id="how-do-i-use-psdeploy">How Do I Use PSDeploy</h2> <p>So! You’ve installed PSDeploy. Now what?</p> <p>At a upper level, you write a *.psdeploy.ps1 deployment script. PSDeploy reads this and runs the deployments that you defined.</p> <p>Let’s step through a few examples to illustrate the basics. We’ll pretend we have the pursuit files and folders for these examples:</p> <p><a href="/images/psdeploy2/sourcefiles.png"><img src="/images/psdeploy2/sourcefiles.png" alt="Source files" /></a></p> <h3 id="a-basic-deployment">A vital deployment</h3> <p>At the very least, we need a Deploy, By, FromSource, and To:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Deploy SomeDeploymentName <span class="o">{</span> By FileSystem <span class="o">{</span> FromSource MyModule To C:\PSDeployTo <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p>You can trammels what PSDeploy sees by running Get-PSDeployment:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Get-PSDeployment -Path C:\PSDeployFrom\my.psdeploy.ps1 </code></pre></div></div> <p><a href="/images/psdeploy2/getpsdeployment.png"><img src="/images/psdeploy2/getpsdeployment.png" alt="Get-PSDeployment" /></a></p> <p>Let’s add some increasingly features!</p> <h3 id="a-more-fun-deployment">A increasingly fun deployment</h3> <p>We’re going to use a few new features of PSDeploy: tags and dependencies. They’re relatively pointless here, but might help you with an very deployment need:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Deploy SomeDeploymentName <span class="o">{</span> By FileSystem AllTheThings <span class="o">{</span> FromSource MyModule, SomeScripts To C:\PSDeployTo DependingOn SomeDeploymentName-MyModule <span class="c1">#DeploymentName-ByName</span> Tagged Dev <span class="o">}</span> By FileSystem MyModule <span class="o">{</span> FromSource MyModule To \\ServerY\c<span class="nv">$\</span>SomePSModulePath, \\ServerX\SomeShare<span class="nv">$\</span>Modules Tagged Prod, Module <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p>Let’s squint at how PSDeploy sees this:</p> <p><a href="/images/psdeploy2/getpsdeployment2.png"><img src="/images/psdeploy2/getpsdeployment2.png" alt="Get-PSDeployment" /></a></p> <p>Like many PowerShell objects, we can find increasingly properties using Select-Object:</p> <p><a href="/images/psdeploy2/getpsdeployment2all.png"><img src="/images/psdeploy2/getpsdeployment2all.png" alt="Get-PSDeployment" /></a></p> <p>So! We see that the deployments have tags, and Get-PSDeployment gave us the results when taking the dependency we specified into account.</p> <p>Finally, maybe we only want to squint at deployments tagged prod:</p> <p><a href="/images/psdeploy2/getpsdeployment2tags.png"><img src="/images/psdeploy2/getpsdeployment2tags.png" alt="Get-PSDeployment" /></a></p> <p>So, how do we unquestionably run these deployments?</p> <h3 id="invoke-psdeploy">Invoke-PSDeploy</h3> <p>Invoking a deployment is simple: we undeniability Invoke-PSDeploy. This borrows a few conventions from Invoke-Pester.</p> <p>Running Invoke-PSDeploy with no parameters will recursively search for *.psdeploy.ps1 files under your current path, and run everything that it finds. In this scenario, any relative paths in the deployment use the current path as the deployment root:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDeploy </code></pre></div></div> <p>Running Invoke-PSDeploy with a path parameter specifying a folder will recursively search for *.psdeploy.ps1 files under that specific folder. It will use the specfied folder as the root for any relative paths in the deployment:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDeploy -Path \\Path\Below\PSDeployConfigs\ </code></pre></div></div> <p>Finally, you can run Invoke-PSDeploy with a path parameter specifying a *.psdeploy.ps1 file. It will use that deployment script’s parent path as the root for any relative paths in the deployment:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-PSDeploy -Path \\Path\To\Some.PSDeploy.ps1 </code></pre></div></div> <h2 id="what-can-i-deploy">What Can I Deploy?</h2> <p>Now that you know how to use PSDeploy, what can you deploy? Some of these things might seem simple, but having a layer of wresting to hibernate some of your lawmaking can be quite helpful:</p> <ul> <li><strong>FileSystem</strong>:Reprintingfiles or folders. Uses copy-item and robocopy overdue the scenes, respectively</li> <li><strong>FileSystemRemote</strong>: Same as FileSystem, but runs in a remote PSSession. Mind the double hop.</li> <li><strong>MkDocs</strong>: Build and deploy an <a href="http://www.mkdocs.org/">MkDocs</a> site - thanks to Michael Lombardi!</li> </ul> <p>It’s pretty yellowish wreck for now, but contributions would be increasingly than welcome. <a href="https://github.com/RamblingCookieMonster/PSDeploy/wiki/Extending-PSDeploy">Extending PSDeploy</a> takes two quick steps: write a script to handle deployments, and a config file that tells PSDeploy what DeploymentType invokes that script.</p> <p>It might be a fun way to get wits working with GitHub, and perhaps some integration testing, if you’re up for it!</p> <p>As time goes on, you can run <code class="highlighter-rouge">Get-PSDeploymentType</code> to list misogynist deployment types, and <code class="highlighter-rouge">Get-PSDeploymentType SomeDeploymentType -ShowHelp</code> to view the help for that deployment type:</p> <p><a href="/images/psdeploy2/getpsdeploymenttype.png"><img src="/images/psdeploy2/getpsdeploymenttype.png" alt="Get-PSDeploymentType" /></a></p> <p><a href="/images/psdeploy2/getpsdeploymenttypehelp.png"><img src="/images/psdeploy2/getpsdeploymenttypehelp.png" alt="Get-PSDeployment" /></a></p> <p>That’s well-nigh it! Let’s recap.</p> <h2 id="what-changed-in-psdeploy">WhatRevertedin PSDeploy?</h2> <ul> <li><strong>*.PSDeploy.ps1 deployment configurations</strong>, in wing to yaml</li> <li><strong>Yaml deployment configurations deprecated</strong>. These still work, but minutiae efforts and new features will focus on *.psdeploy.ps1 scripts</li> <li><strong>Tagged deployments</strong> to help filter or categorize deployments</li> <li><strong>Deployment dependencies</strong> to help ensure deployments perform in the necessary order</li> <li><strong>Noop DeploymentType</strong> to help explore or debug the environment when a deployment runs</li> </ul> <h2 id="wrapping-up">Wrapping Up</h2> <p>This was a fun little project! I’ve found the functionality quite helpful for PowerShell based deployments, whether they start in Jenkins, GitLab CI, or from the shell.</p> <p>If you’d like to contribute or collaborate, <a href="https://github.com/RamblingCookieMonster/PSDeploy">issues and pull requests</a> would be welcome! If you want to proffer PSDeploy with increasingly deployment types, some unenduring notes are misogynist <a href="https://github.com/RamblingCookieMonster/PSDeploy/wiki/Extending-PSDeploy">in the wiki</a>, and I would be increasingly than happy to help!</p> <p>Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/PSDeploy-Take-Two/">PSDeploy&#58; Take Two</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on April 01, 2016.</p> http://ramblingcookiemonster.github.io/Learning-And-Sharing 2016-02-23T22:00:00+00:00 2016-02-23T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a> <ul> <li><a href="#firsttechjob" id="markdown-toc-firsttechjob">FirstTechJob</a></li> </ul> </li> <li><a href="#learning" id="markdown-toc-learning">Learning</a> <ul> <li><a href="#rambling-1" id="markdown-toc-rambling-1">Rambling</a></li> <li><a href="#takeaway" id="markdown-toc-takeaway">Takeaway</a></li> </ul> </li> <li><a href="#sharing" id="markdown-toc-sharing">Sharing</a> <ul> <li><a href="#rambling-2" id="markdown-toc-rambling-2">Rambling</a></li> <li><a href="#takeaway-1" id="markdown-toc-takeaway-1">Takeaway</a></li> </ul> </li> <li><a href="#closing" id="markdown-toc-closing">Closing.</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>Over the weekend, a fun thing happened on Twitter. Someone encouraged folks to post well-nigh their first tech job.</p> <p><a href="https://twitter.com/garethr/status/701099443857309696"><img src="/images/learn/firsttechjob.png" alt="FirstTechJob" /></a></p> <p>It brought to mind two of my primary motivations: learning, and sharing.</p> <h3 id="firsttechjob">FirstTechJob</h3> <p>I lucked out. While I’ve unchangingly been enthusiastic well-nigh computing, when I first unromantic for a job in IT, my wits consisted of an unrelated undergrad stratum (brains!), a co-op in risk and compliance (not very technical), and stuff a student in a networking and systems wardship program at a local university.</p> <p>As much as you learn in school, we all know that nothing can make up for very wits in the real world. I still have no idea what I’m doing, but I had plane less of a track when then.</p> <p>So! I unromantic for a few jobs. I was thesping a help sedentary job would be the only option, probably on an odd shift.</p> <p>I didn’t hear when from most places. Eventually I took a shot in the dark, and unromantic for a data interfaces job at a local hospital; something I had no merchantry applying for. I chatted with the hiring manager, the place seemed nice, but I was unmistakably not the weightier fit. Thankfully, they ended up passing my resume along, and I got a random undeniability for a site support position.</p> <p>That was it! I started working with Windows in a domain environment for the first time, helping folks with their laptops, desktops, and other vendee systems. Somehow, I managed to (mostly) steer well-spoken of printers.</p> <h2 id="learning">Learning</h2> <h3 id="rambling-1">Rambling</h3> <p>Learning is so important in the world of technology. If you don’t enjoy learning, you’re going to hate working in IT. Or you’ll be <em>that guy/girl</em>, the one who <a href="http://ramblingcookiemonster.github.io/Dealing-With-The-Click-Next-Admin/">happily resorts to clicking next</a>, no matter how much encouragement and support you requite them to learn increasingly and help themselves and their team.</p> <p><a href="https://twitter.com/jepayneMSFT/status/701136556338184192"><img src="/images/learn/noteveryone.png" alt="Not everyone" /></a></p> <p>I started out as <em>that guy</em>, increasingly out of ignorance than intention. I tried to eek out my co-workers’ years of wisdom. I learned well-nigh important technologies at each job withal the way. When I moved up to a third level support role, I learned increasingly well-nigh servers and networking, and got a superficial introduction to Active Directory, VMware, and at the suggestion of a mentor, VBscript.</p> <p>That last bit panned out. I moved into a systems engineer role immediately without the exposure to VBscript, probably way surpassing I was ready. So I dove in! VBscript was a pain, but the <a href="https://www.penflip.com/powershellorg/why-powershell/blob/master/chapter2.txt">value of scripting</a> was clear: thankfully PowerShell was once well established!</p> <p>Rather than picking a narrow field to specialize in, I decided to focus on something that applies wideness IT: scripting! I started <a href="http://ramblingcookiemonster.github.io/How-Do-I-Learn-PowerShell/">learning PowerShell</a>, and protract to learn increasingly to this day. One of the many <a href="https://www.penflip.com/powershellorg/why-powershell/blob/master/chapter3.txt">benefits to PowerShell</a> is that you can use it to glue together pretty much any technology. I used this to learn more, and to get involved in interesting projects. SQL Server, Infoblox, Isilon, vCenter, Active Directory, SCOM, you name it.</p> <p>I’m pursuit the same pattern today, and am starting to to learn *nix toolsets and basics, Python, Ruby, Puppet, Salt, and more. The yaks are lining up.</p> <h3 id="takeaway">Takeaway</h3> <p>These helped me out. Take this with a grain of salt, but I suspect they may help you as well:</p> <ul> <li><strong>Be curious</strong>. Explore. If you don’t enjoy learning, steer well-spoken of IT</li> <li><strong><em>It’s not my job</em> is a terrible excuse</strong>.Unchanginglybe unshut to learning topics outside of your current role</li> <li><strong>Find and learn force-multipliers that wield wideness most jobs in IT</strong>. Scripting (e.g. PowerShell), data manipulation (e.g. SQL), and others like these are valuable skills</li> <li><strong>You don’t need to specialize</strong>. Knowing a wholesale range of topics at shallow to intermediate depth, and a few at wide-stretching depth can be quite valuable</li> </ul> <h2 id="sharing">Sharing</h2> <h3 id="rambling-2">Rambling</h3> <p>Learning and sharing go hand in hand. Many of the resources I found most helpful were shared by other IT professionals: lawmaking snippets, functions, tools, libraries, ideas, questions and answers, you name it.</p> <p>So! I tried to contribute back.</p> <p>I started out by just sharing tips, tricks, and <a href="http://ramblingcookiemonster.github.io/Pages/Tools/index.html">tools</a> with co-workers. Eventually I picked up PowerShell, and started sharing snippets, and modules (think of them like toolboxes) with a tuft of handy functions (tools) oriented virtually our environment.</p> <p>I enjoyed sharing with co-workers, so I took it to the next stage, and started a blog. Nothing too exciting, I mostly shared tech news and ideas to start. A first squint at PowerShell WebWanglein the wild. Details on the superstitious new PowerShell 3. A <a href="http://ramblingcookiemonster.github.io/Pages/PowerShellResources/index.html#cheat-sheets-and-quick-references">PowerShell Cheat Sheet</a>. A few months without starting, I released my first function (a fork of Shay Levy’s code), <a href="https://gallery.technet.microsoft.com/scriptcenter/Get-NetworkStatistics-66057d71">Get-NetworkStatistics</a>.</p> <p>I’ve been sharing overly since. If a function or module is unstipulated purpose, doesn’t include merchantry logic, and is hands sanitized, I publish it. I didn’t explicitly ask my boss, although the topic came up a few times. I’d rather ask for forgiveness than permission, but you might have a tighter risk appetite, or work for a increasingly litigious employer.</p> <p>I find that topic irksome. Employers goody from increasingly informed IT practitioners. Preventing IT folks from sharing their work holds when both the employer and IT practitioners who could have benefited from the knowledge. It moreover creates some uncertainty, where the increasingly risk-averse among us stave sharing, plane if they aren’t under contractual obligation to hoard knowledge.</p> <p>We all goody from sharing. Rather than having every IT professional write indistinguishable tools, with no feedback or contributions from others, it makes increasingly sense to share our lawmaking and ideas. Less duplication of effort, increasingly vision on the lawmaking or ideas, increasingly feedback from folks taking your ideas or lawmaking and using them in new and variegated ways; this benefits the practice of IT itself. Heck, it might not be the intent, but if you read the LOPSA lawmaking of ethics, they mention how important sharing is:</p> <p><a href="https://lopsa.org/CodeOfEthics"><img src="/images/learn/lopsa.png" alt="LOPSA Education" /></a></p> <h3 id="takeaway-1">Takeaway</h3> <p>Again, I found these beneficial, but take them with a grain of salt:</p> <ul> <li>Share. <strong>You’ll learn more</strong>. You might polish your lawmaking and follow practices you would otherwise ignore. You’ll probably want to review information and learn increasingly well-nigh the topic at hand surpassing you post well-nigh it</li> <li>Share. <strong>You might get helpful feedback</strong> and end up with a largest solution</li> <li>Share. <strong>You’ll help others in IT</strong>, who in turn might share their own ideas and code</li> <li>Share. <strong>It looks good to many employers</strong>. All else stuff equal, if I’m looking at two candidates, and only one of them has lawmaking and ideas unshut to the public, they’ll probably get the nod</li> <li><strong>Don’t rule out sharing off-hand</strong>. Consider the risk and reward of sharing your sanitized lawmaking and ideas</li> <li><strong>Provide digestible details</strong>. What’s the goal of your code? How do we install it? How do we use it? Readme.md files or short blog posts go a long way. As silly as it sounds, turned-on gifs are a unconfined way to quickly get wideness an idea.</li> </ul> <h2 id="closing">Closing.</h2> <p>That’s well-nigh it!</p> <p>I owe a debt of gratitude to my co-workers, and the various PowerShell and other IT communities and contributors out there. If they hadn’t shared their knowledge, I wouldn’t have had the same resources to learn with, and wouldn’t be worldly-wise to share when the little that I can.</p> <p>Don’t be wrung of learning something new, and share what you’re learning withal the way. I went from “what’s PowerShell?” to “<a href="https://www.penflip.com/powershellorg/why-powershell/blob/master/chapter1.txt">Why PowerShell?</a>.” in a few short years. Years sounds like a long time, but spending a little time every day was all it took.</p> <p>Cheers! Hopefully when to posting increasingly technical content soon.</p> <p><a href="http://ramblingcookiemonster.github.io/Learning-And-Sharing/">On Learning, Sharing, and My First Tech Job</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on February 23, 2016.</p> http://ramblingcookiemonster.github.io/PowerShell-Is-Too-Hard 2016-02-18T22:00:00+00:00 2016-02-18T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#the-task" id="markdown-toc-the-task">The Task</a> <ul> <li><a href="#read-a-file-in-c" id="markdown-toc-read-a-file-in-c">Read a file in C</a></li> <li><a href="#read-a-file-in-c-1" id="markdown-toc-read-a-file-in-c-1">Read a file in C#</a></li> <li><a href="#read-a-file-in-python" id="markdown-toc-read-a-file-in-python">Read a file in Python</a></li> <li><a href="#read-a-file-in-powershell" id="markdown-toc-read-a-file-in-powershell">Read a file in PowerShell</a></li> </ul> </li> <li><a href="#you-dont-need-to-be-a-developer-to-use-powershell" id="markdown-toc-you-dont-need-to-be-a-developer-to-use-powershell">You Don’t Need to be a Developer to use PowerShell</a> <ul> <li><a href="#how-do-i-get-started" id="markdown-toc-how-do-i-get-started">How do I get started?</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h2 id="rambling">Rambling</h2> <p>I lucked out and no longer need to convince co-workers or their managers that it’s in their weightier interest to learn how to write simple code. Unfortunately, in most enterprise environments, a <a href="http://ramblingcookiemonster.github.io/Dealing-With-The-Click-Next-Admin/">significant proportion</a> of the IT workforce seem to enjoy square wheels.</p> <p>How many times have you seen this?</p> <p><a href="/images/hard/toobusy.png"><img src="/images/hard/toobusy.png" alt="I don't have time to learn" /></a></p> <p>Some worldwide excuses include <em>it’s too hard</em>, <em>I’m too busy</em>, or <em>I’m not a developer!</em> Often with context virtually how rented they are working on transmission tasks that could be automated, or simplified with a small script.</p> <p>I won’t tell you that going from no wits to a decent scripter is something you learn overnight, but you need to start somewhere.</p> <p>PowerShell is a tool that most of us can handle learning, and would goody from greatly, plane right out of the gate. Lower level languages are unconfined for developers, but they might require a increasingly significant investment in time, without as much value in return (for an IT professional).</p> <p>Let’s illustrate this difference and highlight how helpful PowerShell is, by looking at a simple task wideness four popular languages.</p> <h2 id="the-task">The Task</h2> <p>We’re going to learn how to read a file. Exciting, right? We’ll do this using the pursuit languages:</p> <ul> <li><strong>PowerShell</strong>, a task-oriented scripting and shell language. Work in a Microsoft ecosystem? <a href="https://www.penflip.com/powershellorg/why-powershell/blob/master/chapter1.txt">PowerShell is probably the weightier way to get things done</a></li> <li><strong>Python</strong>, a high-level unstipulated purpose programming language</li> <li><strong>C#</strong>, a feature-full unstipulated purpose programming language</li> <li><strong>C</strong>, a low-level language that terrifies me</li> </ul> <p>This is roughly in order of how much hand holding a language will do for you, which many IT professionals don’t realize. Let’s start from the ground up.Alimonyin mind there are many ways to skin a cat in each of these languages.</p> <h3 id="read-a-file-in-c">Read a file in C</h3> <p>So! First, we need to get a compiler (Visual Studio is your weightier bet). Then, we search virtually for a simple solution. Protip: don’t write in C if you can help it. You need to do everything on your own, which invariably ways whatever you write will be insecure.</p> <p>Here’s a “simple” program to read C:\file.txt:</p> <script src="https://gist.github.com/6170da1a25751fe3b111.js"> </script> <p><a href="/images/hard/c.png"><img src="/images/hard/c.png" alt="C" /></a></p> <p>And voila, it worked!</p> <p>Good luck querying LDAP, pulling info from VMware vCenter, or hitting a web API using C. Also, get ready for fun if you’re not very familiar with C and need to read or debug someone else’s code.</p> <p>Let’s move up to a higher level language, C#.</p> <h3 id="read-a-file-in-c-1">Read a file in C#</h3> <p>Next up, C#. You might have Visual Studio installed already; if not you can compile a simple C# file with csc.exe: e.g. <code class="highlighter-rouge">C:\Windows\Microsoft.NET\Framework\v4.0.30319\csc.exe YourFile.cs</code></p> <script src="https://gist.github.com/6c3ae82a321b792b7cca.js"> </script> <p><a href="/images/hard/cs.png"><img src="/images/hard/cs.png" alt="C#" /></a></p> <p>Okay! Now we’re getting somewhere.Planeif I couldn’t write that on my own, I can get a finger for what’s going on by reading it. But, it’s still a bit heavy on the developer side:</p> <ul> <li>You need to compile</li> <li>You can’t forget a semi-colon</li> <li>You’re not just saying “read-a-file”, you have to use .NET classes to tell C# how to read a file. Now do this for every task in your solution. This is why I find comments like <em>just use C# instead of PowerShell</em> quite absurd</li> </ul> <p>If you’re not a developer, it’s important to note how helpful that last bullet is. While C# isn’t a task-oriented language, it uses the .NET Framework, which provides a vast and incredibly valuable library of code.</p> <p>If you’re an IT professional trying to automate a technology where the vendor got lazy, there’s a good endangerment you can write a PowerShell function or module that wraps their .NET library (the caveats <a href="http://ramblingcookiemonster.github.io/REST-PowerShell-and-Infoblox/">here</a> still apply: vendors should really provide a PowerShell module for their customers).</p> <p>Enough rambling, let’s move into the world of scripting, and squint at reading a file in Python.</p> <h3 id="read-a-file-in-python">Read a file in Python</h3> <p>So!Stuffin the Microsoft ecosystem, PowerShell is often the top choice. If you plan to step outside that ecosystem, or just want the wits (protip: this is very valuable), Python is a solid, outgoing choice.</p> <p>We’re going to cut our line count quite a bit:</p> <script src="https://gist.github.com/bacdf9bd418f5960fe0f.js"> </script> <p><a href="/images/hard/python.png"><img src="/images/hard/python.png" alt="Python" /></a></p> <p>Nice! So that was much shorter, and the result is plane easier to read than C#. Python is a upper level language often used for scripting, but it’s not as visionary out and task-based as PowerShell.</p> <h3 id="read-a-file-in-powershell">Read a file in PowerShell</h3> <p>We wrap up with the simplest solution:</p> <pre><code class="language-PowerShell">Get-Content C:\file.txt </code></pre> <p><a href="/images/hard/powershell.png"><img src="/images/hard/powershell.png" alt="PowerShell" /></a></p> <p>It all boils lanugo to a single, easy to read command. If your co-workers mutter that they aren’t developers, ask them if they find commands like the one whilom too difficult. Hopefully they are unshut to learning, and you can help get them started. Otherwise, what are they doing in IT?</p> <p>A developer could hop between any of the languages whilom and more, for a variety of tasks, often with significant expertise in one or two of their favorites. Learning the nuts of a task-based language is not <em>too hard,</em> and now that nearly everything has an API, <a href="http://everythingsysadmin.com/2014/02/do-system-administrators-need-.html">warnings like this</a> will be increasingly than just wishful thinking.</p> <p>So! What’s the moral of this story anyways?</p> <h2 id="you-dont-need-to-be-a-developer-to-use-powershell">You Don’t Need to be a Developer to use PowerShell</h2> <p>Anyone in a Microsoft ecosystem should pick up the nuts (or beyond) of PowerShell. If you can requite flipside person instructions - maybe read a recipe, or describe how to troubleshoot an iPhone - you once have the vital ingredients of coding. The rest is just syntax, which is a quick Google search away, or plane provided to you with <a href="https://blogs.technet.microsoft.com/heyscriptingguy/2014/01/25/using-powershell-ise-snippets-to-remember-tricky-syntax/">snippets</a>.</p> <p>In fact, PowerShell is a prime candidate for scripting and automation <a href="http://ramblingcookiemonster.github.io/PowerShell-Beyond-Administration/">outside of the world of IT</a>. How would we get there?</p> <ul> <li>An unshut source and cross-platform implementation of the .NET Framework (check) and PowerShell (still waiting)</li> <li>Microsoft pushing their own product groups to provide an interface through PowerShell, for both wardship and working with products (SQL is a prime example), and vastitude their Server and Tools organization</li> <li>Microsoft pushing and profitable third parties in developing PowerShell modules to imbricate a wider variety of technologies</li> <li>Microsoft and the polity raising and pushing the PowerShell Gallery - ~560 modules compared to CPAN, PyPI, or RubyGems is sad, regardless of the throne start these have</li> <li>Luck : )</li> </ul> <h3 id="how-do-i-get-started">How do I get started?</h3> <p>So! Until then… Are you focused on desktops / clients, printers, identity and wangle management, databases, virtualization, servers, security, communications, support, networking, or any other pursuit in a Microsoft ecosystem? You should probably learn the nuts of PowerShell.</p> <p>References abound, I unchangingly recommend <a href="http://ramblingcookiemonster.github.io/How-Do-I-Learn-PowerShell/">a three pronged approach</a>:</p> <ul> <li><a href="http://ramblingcookiemonster.github.io/How-Do-I-Learn-PowerShell/#formal-resources">Pick up a formal resource</a>. Maybe a typesetting you read over a month of lunches.</li> <li><a href="http://ramblingcookiemonster.github.io/How-Do-I-Learn-PowerShell/#join-the-community">Join the community</a>. Learn from existing tools and scripts. Ask questions and listen in on <a href="https://www.reddit.com/r/PowerShell/comments/2gm75d/what_are_some_of_your_favorite_powershell_blogs/ckkfqel">Twitter</a>, <a href="http://slack.poshcode.org/">Slack</a>, and <a href="http://powershell.org/">PowerShell.org</a>. Stop by a local PowerShell user group. Contribute to all of these if you can!</li> <li><a href="http://ramblingcookiemonster.github.io/How-Do-I-Learn-PowerShell/#spend-some-time-with-powershell">Use PowerShell every day</a>.Planeif only for a few minutes. You can’t learn this in one shot. You won’t learn this if you don’t use it regularly.</li> </ul> <p>Hope to see you virtually the community!</p> <p><a href="http://ramblingcookiemonster.github.io/PowerShell-Is-Too-Hard/">PowerShell Is Too Hard</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on February 18, 2016.</p> http://ramblingcookiemonster.github.io/Cross-Platform-PowerShell-Remoting 2016-02-06T22:00:00+00:00 2016-02-06T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a> <ul> <li><a href="#the-options" id="markdown-toc-the-options">The Options</a></li> <li><a href="#ssh" id="markdown-toc-ssh">SSH</a> <ul> <li><a href="#getting-started-with-ssh-on-windows" id="markdown-toc-getting-started-with-ssh-on-windows">Getting Started with SSH on Windows</a></li> <li><a href="#running-powershell-from-ssh" id="markdown-toc-running-powershell-from-ssh">Running PowerShell from SSH</a></li> </ul> </li> <li><a href="#winrb--winrm" id="markdown-toc-winrb--winrm">WinRb / WinRM</a> <ul> <li><a href="#using-winrm-from-ruby" id="markdown-toc-using-winrm-from-ruby">Using WinRM From Ruby</a></li> </ul> </li> <li><a href="#which-should-i-use" id="markdown-toc-which-should-i-use">Which Should I Use?</a></li> <li><a href="#whats-on-the-horizon" id="markdown-toc-whats-on-the-horizon">What’s on the Horizon?</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h1 id="rambling">Rambling</h1> <p>Wow! Four months since my last post. New city, new home, new job, new tiny human in less than a month, and thankfully, same Wegmans.</p> <p>I lucked out on the job front. Found a fun team full of talented folks using modern tools, with a role focusing on Microsoft and related tech, in a predominantly *nix / OSS oriented environment. Plenty of room to learn and explore!</p> <p>Of course, with everyone stuff well-versed in languages like whack and Python, and with most infrastructure residing on *nix systems using tools like Puppet for configuration, learning PowerShell hasn’t made it to their to-do list. Yet. Hopefully that changes - from my perspective, nothing comes tropical to PowerShell as an approachable, task-oriented glue language - imagine if it were unshut sourced and ported to other systems?</p> <p>Until then, rather than miss out on the goodness PowerShell enables, we need to icon out an tideway to invoke PowerShell from navigate platform systems!</p> <h2 id="the-options">The Options</h2> <p>There are a number of approaches, each with benefits and drawbacks. At a upper level, we could use…</p> <ul> <li>Remoting solutions like SSH or WinRM</li> <li>API oriented solutions like <a href="https://github.com/toenuff/flancy">flancy</a>, or something heavier like ASP.NET Web API</li> <li>Web based solutions, from tools like rundeck, to shoe-horning <a href="https://hodgkins.io/automating-with-jenkins-and-powershell-on-windows-part-1">Jenkins</a>, to <a href="https://github.com/vmware/webcommander">WebCommander</a>, to towers your own ASP.NET / C# page</li> </ul> <p>The other options are veritably worth exploring, but a remoting solution would be very flexible with minimal overhead. So, how can we use WinRM or SSH to undeniability PowerShell from non-Windows systems?</p> <h2 id="ssh">SSH</h2> <p>Wait, an SSH server on Windows? Do you really want to rely on some third party or outdated unshut source option?</p> <p>Turns out Microsoft is <a href="https://blogs.msdn.microsoft.com/powershell/2015/06/03/looking-forward-microsoft-support-for-secure-shell-ssh/">planning</a> to unhook an SSH vendee <em>and server</em> for Windows systems, and that there is once a usable (ish) <a href="https://github.com/PowerShell/Win32-OpenSSH">early preview</a>.</p> <p>Given that it’s so early, you should definitely consider testing this out and <a href="https://github.com/PowerShell/Win32-OpenSSH/issues">contributing feedback</a>. You might help shape diamond decisions and modernize the solution for you, and everyone else using it.</p> <h3 id="getting-started-with-ssh-on-windows">Getting Started with SSH on Windows</h3> <p>This is the easy part! Just follow <a href="https://github.com/PowerShell/Win32-OpenSSH/wiki/Install-Win32-OpenSSH">the instructions here</a>. At some point, this will hopefully receive CI/CD treatment to unhook stable and dev releases, perhaps to a package manager we can hit from <a href="https://github.com/OneGet/oneget">OneGet</a>.</p> <p>If you’re reading this in early 2016, be sure to <a href="https://github.com/PowerShell/Win32-OpenSSH/releases">grab the right architecture</a>; the link in the instructions will point you to the 32-bit release.</p> <h3 id="running-powershell-from-ssh">Running PowerShell from SSH</h3> <p>I ran into no issues getting up and running <em>with non-interactive commands</em>. Let’s squint at a few examples:</p> <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Run ipconfig</span> ssh <span class="nt">-l</span> wframe@my.domain ts1 ipconfig </code></pre></div></div> <p><a href="/images/ssh/ipconfig.png"><img src="/images/ssh/ipconfig.png" alt="SSH ipconfig" /></a></p> <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># PowerShell isn't the default shell. Let's try it out.</span> ssh <span class="nt">-l</span> wframe@my.domain ts1 powershell.exe <span class="nt">-command</span> <span class="s2">"get-service sshd"</span> </code></pre></div></div> <p><a href="/images/ssh/ssh-get-service.png"><img src="/images/ssh/ssh-get-service.png" alt="SSH service" /></a></p> <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Double hop issue? Nope!</span> ssh <span class="nt">-l</span> wframe@my.domain ts1 powershell.exe <span class="nt">-command</span> <span class="s2">"get-aduser wframe"</span> </code></pre></div></div> <p><a href="/images/ssh/ssh-aduser.png"><img src="/images/ssh/ssh-aduser.png" alt="SSH aduser" /></a></p> <p>Nice! So, SSHD is working, we can run PowerShell.exe, and we don’t have to worry well-nigh the double-hop issue that you might be familiar with on the PowerShell remoting / Kerberos side!</p> <p>How well-nigh something a little longer, with variables?</p> <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ssh <span class="nt">-l</span> wframe@rc.domain ts1 powershell.exe <span class="nt">-noprofile</span> <span class="nt">-command</span> <span class="s2">" </span><span class="nv">$Service</span><span class="s2"> = 'SSHD' Get-Service </span><span class="nv">$Service</span><span class="s2"> "</span> </code></pre></div></div> <p><a href="/images/ssh/ssh-escape-char-fail.png"><img src="/images/ssh/ssh-escape-char-fail.png" alt="SSH escape char fail" /></a></p> <p>Here’s where we run into our first issue. When calling PowerShell from whack or flipside interpreter, you need to be mindful of all the interpreters. In our case, whack helpfully replaced $service for us.Withoutbash, presumably cmd.exe needs to be rumored for (i.e. we’re not dropped right into PowerShell), and then PowerShell itself. Maybe you have a snippet that needs double quotes: have fun!</p> <p>Ideally, you manage Windows systems from Windows systems that are once running PowerShell, and stave the mess, but you can’t unchangingly make that call.</p> <p>In this example, we can simply escape the $:</p> <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ssh <span class="nt">-l</span> wframe@rc.domain ts1 powershell.exe <span class="nt">-noprofile</span> <span class="nt">-command</span> <span class="s2">" </span><span class="se">\$</span><span class="s2">Service = 'SSHD' Get-Service </span><span class="se">\$</span><span class="s2">Service "</span> </code></pre></div></div> <p><a href="/images/ssh/ssh-escape-char.png"><img src="/images/ssh/ssh-escape-char.png" alt="SSH escape char" /></a></p> <p>Now, you could unchangingly utopian this into a shell script, perhaps using expect to handle the password, but ideally we could wrap this in Python or Ruby. I had no luck getting this to work in <a href="https://github.com/paramiko/paramiko">Paramiko</a> or it’s abstraction, <a href="https://pypi.python.org/pypi/spur">spur</a>, but finger self-ruling to requite it a try, and be sure to let others know if you have any luck!</p> <p>That’s well-nigh it - let’s move on and squint at using WinRM.</p> <h2 id="winrb--winrm">WinRb / WinRM</h2> <p>There are a number of ways to invoke PowerShell over WinRM from other languages. For many, the Python and Ruby modules will likely be your first stop.</p> <p>We’re going to skip the Python pywinrm module, given that it requires either plaintext auth, Kerberos, or fun with SSL. If you once use Kerberos or have a PKI infrastructure in place, you could safely use this. Otherwise, you’ll find a number of guides for <a href="https://github.com/diyan/pywinrm">pywinrm</a>, <a href="https://github.com/WinRb/WinRM">WinRb</a>, and more, instructing you to configure WinRM to enable vital auth and AllowUnencrypted. <em>Don’t</em>.</p> <p>We’ll use the Ruby module for WinRM. Install Ruby on your system, <code class="highlighter-rouge">gem install -r winrm</code>, and yonder we go!</p> <h3 id="using-winrm-from-ruby">Using WinRM From Ruby</h3> <p>We’ll seem you once have have PowerShell remoting enabled and wieldy from flipside Windows system. Thanks to Dan Wanek and Matt Wrock, we can now use <a href="http://www.hurryupandwait.io/blog/sane-authenticationencryption-arrives-to-ruby-based-cross-platform-winrm-remote-execution">NTLM authentication</a>, which doesn’t come with the dependencies or complexities of Kerberos or a PKI.</p> <p>Security note: Do consider using SSL and/or a increasingly modern hallmark mechanism like Kerberos, but for quick and dirty testing, NTLM is a far safer bet than vital auth with AllowUnencrypted. Would love to hear some real security folks weigh in on the current method…</p> <p>Let’s get started!</p> <p>Here’s the gist of what I might run.Increasinglyexamples abound, just alimony in mind we’re using create_executor’s run_powershell_script method, not the <a href="https://github.com/WinRb/WinRM#deprecated-methods">deprecated alternatives</a>.</p> <div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#!/usr/local/bin/ruby</span> <span class="nb">require</span> <span class="s1">'winrm'</span> <span class="n">winrm</span> <span class="o">=</span> <span class="no">WinRM</span><span class="o">::</span><span class="no">WinRMWebService</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'http://ts1:5985/wsman'</span><span class="p">,</span> <span class="ss">:negotiate</span><span class="p">,</span> <span class="ss">:user</span> <span class="o">=&gt;</span> <span class="s1">'wframe'</span><span class="p">,</span> <span class="ss">:pass</span> <span class="o">=&gt;</span> <span class="s1">'my password'</span><span class="p">)</span> <span class="n">winrm</span><span class="p">.</span><span class="nf">create_executor</span> <span class="k">do</span> <span class="o">|</span><span class="n">executor</span><span class="o">|</span> <span class="n">executor</span><span class="p">.</span><span class="nf">run_powershell_script</span><span class="p">(</span><span class="s2">"get-service 'sshd'"</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">stdout</span><span class="p">,</span> <span class="n">stderr</span><span class="o">|</span> <span class="no">STDOUT</span><span class="p">.</span><span class="nf">print</span> <span class="n">stdout</span> <span class="no">STDERR</span><span class="p">.</span><span class="nf">print</span> <span class="n">stderr</span> <span class="k">end</span> <span class="k">end</span> </code></pre></div></div> <p><a href="/images/ssh/winrm-simple.png"><img src="/images/ssh/winrm-simple.png" alt="WinRM service" /></a></p> <p>This is my first time touching Ruby outside of some sparing use with the GitHub pages setup overdue this blog, but I figured it would be worth trying to utopian this into a simple, re-usable function. Cobbled <a href="https://gist.github.com/RamblingCookieMonster/a1645b534fed02b4b368">this</a> together from various stackoverflow queries; feedback or tips welcome.</p> <p>Let’s kick the tires a bit:</p> <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Double hop issue? Yep!</span> ps-winrm.rb <span class="nt">-u</span> wframe <span class="nt">-c</span> <span class="s2">"get-aduser wframe"</span> </code></pre></div></div> <p><a href="/images/ssh/winrm-aduser-fail.png"><img src="/images/ssh/winrm-aduser-fail.png" alt="WinRM aduser fail" /></a></p> <p>The error message is a bit garbled, but you can get the gist of it. While the underlying rationalization differs, we run into the double hop issue PowerShell remoting typically gives us.</p> <p>Given that NTLM support was just implemented, and that a skin-deep search doesn’t turn up anything on CredSSP, it looks like the usual CredSSP workaround won’t help.</p> <p>Security note: CredSSP will make a red team happy. Do consider <a href="http://www.powershellmagazine.com/2014/03/06/accidental-sabotage-beware-of-credssp/">the security implications</a>, as well as the fact that if you’re using RDP, you’re once using CredSSP.</p> <p>Another workaround that works here would be to live dangerously and hit a domain controller directly:</p> <p><a href="/images/ssh/winrm-aduser.png"><img src="/images/ssh/winrm-aduser-dc.png" alt="WinRM aduser" /></a></p> <p>That’s well-nigh it! I have seen some odd issues with commands that use runspaces, but most every-day use cases seem to work.</p> <h2 id="which-should-i-use">Which Should I Use?</h2> <p>Both options have their strengths and weaknesses, this is really up to you. I personally prefer to use languages like Ruby or Python over bash, and SSHD is quite new, so I’ll likely lean towards WinRM in the interim. Once I can get Paramiko or spur working versus Microsoft’s SSHD, I’ll likely switch over to that.</p> <p>Looking ahead, SSH on Windows seems very promising, and should enable simpler integration with navigate platform solutions.</p> <p>The WinRM implementations are worth exploring and will protract improving, but will come with some limitations, and don’t seem to be receiving contributions from Microsoft (yet).</p> <h2 id="whats-on-the-horizon">What’s on the Horizon?</h2> <p>A nice side-effect of Microsoft using GitHub is that we get a glimpse of what’s coming lanugo the road.</p> <p>Check out the Win32-OpenSSH <a href="https://github.com/PowerShell/Win32-OpenSSH/issues/">issues</a> - you’ll see some notes on current limitations and bugs, withal with some interesting ideas and plans. Curious to see how they’ll implement key based hallmark for domain finance (<a href="https://github.com/PowerShell/Win32-OpenSSH/issues/39">issue 39</a>).</p> <p>Be sure to kick the wheels of Microsoft’s SSHD implementation. You can unquestionably contribute, whether you submit a pull request, a thoughtful bug report, or an idea for a feature. It’s still quite early, so if you get in your ideas now, who knows, you might help modernize this for everyone!</p> <p>Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/Cross-Platform-PowerShell-Remoting/">Cross Platform PowerShell Remoting</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on February 06, 2016.</p> http://ramblingcookiemonster.github.io/Evil-LDAP-Queries 2015-10-02T22:00:00+00:00 2015-10-02T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#the-fun-begins" id="markdown-toc-the-fun-begins">The Fun Begins</a></li> <li><a href="#finding-expensive-inefficient-or-long-running-ldap-queries" id="markdown-toc-finding-expensive-inefficient-or-long-running-ldap-queries">Finding Expensive, Inefficient, or Long Running LDAP Queries</a> <ul> <li><a href="#prerequisites" id="markdown-toc-prerequisites">Prerequisites</a></li> <li><a href="#the-registry" id="markdown-toc-the-registry">The Registry</a></li> <li><a href="#collecting-the-results" id="markdown-toc-collecting-the-results">Collecting the Results</a></li> <li><a href="#parsing-and-interpreting-the-results" id="markdown-toc-parsing-and-interpreting-the-results">Parsing and Interpreting the Results</a></li> </ul> </li> <li><a href="#powershell-modules-write-them" id="markdown-toc-powershell-modules-write-them">PowerShell Modules: Write Them</a></li> <li><a href="#get-on-with-it" id="markdown-toc-get-on-with-it">Get On With It!</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h1 id="rambling">Rambling</h1> <p>A while back, I noticed an intriguing tweet from Mark Morowczynski at Microsoft:</p> <p><img src="/images/ldap/tweet.png" alt="tweet" /></p> <p>Long story short, Mark explains how to search for and unriddle <a href="http://blogs.technet.com/b/askpfeplat/archive/2015/05/11/how-to-find-expensive-inefficient-and-long-running-ldap-queries-in-active-directory.aspx">expensive, inefficient, and long running LDAP queries in Active Directory</a>. Bookmarked.</p> <p>This week, Mark’s vendible came in quite handy.</p> <h1 id="the-fun-begins">The Fun Begins</h1> <p>Wednesday morning, someone asked well-nigh some SCOM alerts for AD. We see these every so often, and I started explaining that sites with one or two domain controllers lead to somewhat worldwide and often innocuous AD Site Performance Health Degraded alerts.</p> <p>I don’t like assumptions though. Interesting. All the domain controllers in one site are at 100% CPU, with crazy CPU queues. This wasn’t a site getting disconnected, or a one-off evil LDAP query. Looks like we get to swoop into Mark’s post!</p> <h1 id="finding-expensive-inefficient-or-long-running-ldap-queries">Finding Expensive, Inefficient, or Long Running LDAP Queries</h1> <h2 id="prerequisites">Prerequisites</h2> <p>Okay! I know I saw ‘PowerShell’ when skimming the article, this should be quick and easy, or so I thought.</p> <p>The final stages of our 2012 R2 roll out are scheduled for next week, so we need to <a href="https://support.microsoft.com/en-us/kb/2800945/en-us">install KB2800945</a> on any remaining 2008 R2 domain controllers, and reboot.</p> <p>AD is somewhat important, and a juicy target - if you have a tight ship and this is a rare scenario, consider opening a ticket with Microsoft and triggering your (security) incident response process here.</p> <h2 id="the-registry">The Registry</h2> <p>Once our prerequisites are in place, we need to flip a few values in the registry.</p> <p>Oh. There’s a screen shot of the registry in the blog post. And a table with registry keys and values. Guess we’ll write <a href="https://github.com/RamblingCookieMonster/PSLDAPQueryLogging/blob/master/PSLDAPQueryLogging/Public/Enable-LDAPQueryLogging.ps1">a PowerShell function</a>!</p> <p>I’m all for quickly fighting fires with a GUI, but I realize I don’t want to deal with enabling and disabling this manually on several machines. Writing a quick POC function will be far faster. Particularly given that this is a simple registry change, and that Shay Levy has written the fantastic <a href="https://psremoteregistry.codeplex.com/">PSRemoteRegistry</a>, which I no longer consider a dependency. Just say no to providers that don’t let you hit remote systems.</p> <p>All we do here is enable logging, and set a few parameters that Mark suggested.</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">Set</span>-RegDWord -ComputerName <span class="nv">$Computer</span> -Hive LocalMachine -Key <span class="s1">'System\CurrentControlSet\Services\NTDS\Diagnostics'</span> -Value <span class="s1">'15 Field Engineering'</span> -data 5 <span class="nb">Set</span>-RegDWord -ComputerName <span class="nv">$Computer</span> -Hive LocalMachine -Key <span class="s1">'SYSTEM\CurrentControlSet\Services\NTDS\Parameters'</span> -Value <span class="s1">'Expensive Search Results Threshold'</span> -data 0 <span class="nb">Set</span>-RegDWord -ComputerName <span class="nv">$Computer</span> -Hive LocalMachine -Key <span class="s1">'SYSTEM\CurrentControlSet\Services\NTDS\Parameters'</span> -Value <span class="s1">'Inefficient Search Results Threshold'</span> -data 0 <span class="nb">Set</span>-RegDWord -ComputerName <span class="nv">$Computer</span> -Hive LocalMachine -Key <span class="s1">'SYSTEM\CurrentControlSet\Services\NTDS\Parameters'</span> -Value <span class="s1">'Search Time Threshold (msecs)'</span> -data 100 </code></pre></div></div> <p>That’s it! My domain controllers are now logging event 1644, with details on each LDAP query that meets the thresholds I just set; in this case, anything taking over 100ms. Time for coffee.</p> <h2 id="collecting-the-results">Collecting the Results</h2> <p>This could be outdated or unappetizing out ignorant knowledge, but I recall wevtutil epl stuff incredibly fast, as compared to reading events using Get-WinEvent (or Get-EventLog, eek!). I’m a fan of using tools that meet your needs. In this case, quickly exporting an evtx is a simple one liner.</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Invoke-Command -ComputerName <span class="nv">$Computer</span> <span class="o">{</span>wevtutil epl <span class="s1">'Directory Service'</span> <span class="s2">"\\</span><span class="nv">$Using</span><span class="s2">:Computer\c</span><span class="nv">$\$ENV</span><span class="s2">:ComputerName-Evil.evtx"</span><span class="o">}</span> </code></pre></div></div> <p>Alrighty, the events are there. Oh. Be sure your Directory Services event log is large unbearable to reservation unbearable data. Maybe <a href="https://github.com/PowerShell/xWinEventLog">use DSC</a>.</p> <p>At this point, I <a href="https://github.com/RamblingCookieMonster/PSLDAPQueryLogging/blob/master/PSLDAPQueryLogging/Public/Disable-LDAPQueryLogging.ps1">reverse</a> the registry changes I made, and get ready for some fun with Excel!</p> <h2 id="parsing-and-interpreting-the-results">Parsing and Interpreting the Results</h2> <p>Mark links to <a href="https://gallery.technet.microsoft.com/scriptcenter/Event-1644-reader-Export-45205268">a script</a> that helps read Event 1644 data. This is quite handy, and it’s superstitious that folks are trying to use PowerShell, but the methodology leaves a bit to be desired.</p> <ul> <li>Install Excel, so that you can wangle the COM interface. <a href="http://ramblingcookiemonster.github.io/PSExcel-Intro/">Sigh</a>.</li> <li>Read the script and Get-Help. No parameters. Sigh.</li> <li>Run the script, follow the Read-Host guide. Sigh.</li> <li>If you’re like many folks out there, you might get a few errors. My output was still usable.</li> </ul> <p>At this point, you can scan virtually the results in Excel, <a href="http://blogs.technet.com/b/askpfeplat/archive/2015/05/11/how-to-find-expensive-inefficient-and-long-running-ldap-queries-in-active-directory.aspx#Analyzing"> as Mark explains in his post</a>.</p> <p>In our case, the TopTime-IP section came in handy. A single system is making a huge portion of these queries, including a number of 8+ second queries using UID, which isn’t indexed. I gleefully dangle my pitchfork and run a reverse lookup.</p> <p><img src="/images/ldap/wat.png" alt="Wat" /></p> <p>Hopefully this pointed you in the right direction or right at the culprit. Depending on what you do or don’t find, there are a variety of helpful resources:</p> <ul> <li><a href="http://blogs.technet.com/b/askpfeplat/archive/2012/11/11/mcm-active-directory-indexing-for-the-masses.aspx">Indexing</a>. Active Directory sits on a database. Like most database engines, your domain controllers will be much, much happier if queries use an indexed attribute.</li> <li>Developers, vendors, and sysadmins might not be familiar with how evil the queries they write can be. <a href="https://msdn.microsoft.com/en-us/library/ms808539.aspx">Long story</a> short? Be as specific as possible. Start as deep as you can, set your search telescopic as tight as possible, and follow efficient query practices like using indexed attributes, returning only the nature you need, and keeping your wild cards at the end of a string, not the start.</li> <li>Other tools, like the <a href="http://blogs.technet.com/b/askds/archive/2010/06/08/son-of-spa-ad-data-collector-sets-in-win2008-and-beyond.aspx">AD data collector sets</a>, <a href="http://blogs.technet.com/b/yongrhee/archive/2014/02/01/tool-server-performance-advisor-spa-v3-1-to-troubleshoot-high-cpu-in-lsass-in-ad-domain-controllers.aspx">SPA</a>, WireShark, or <a href="https://technet.microsoft.com/en-us/library/jj714801.aspx">Microsoft Message Analyzer</a> might come in handy.</li> <li>We’re thesping you ruled out other worldwide causes, like non-LSASS offenders, mis-sized domain controllers, and other scenarios. Google and MSFT resources are your friend : )</li> </ul> <p>Now, what well-nigh next time?</p> <h1 id="powershell-modules-write-them">PowerShell Modules: Write Them</h1> <p><a href="http://powershell.org/wp/2015/08/16/abstraction-and-configuration-data/">Abstraction is important</a>. Don’t think in terms of “writing scripts”, try to think in terms of writing re-usable tools: functions. Once you’re comfy writing these tools, you need a toolbox to put them in: modules.</p> <p>Want to forget well-nigh tools and toolboxes to an extent? <a href="https://channel9.msdn.com/Series/Getting-Started-with-PowerShell-Desired-State-Configuration-DSC">Desired State Configuration</a> and configuration management solutions that layer on top might be up your alley. But I digress, and it’s important to note that configuration management isn’t a panacea; Do you plan to transpiration your configurations for a matriculation of system just to enable diagnostics on a handful of instances? Nope.</p> <p>I used a quick <a href="http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module/">recipe for towers a PowerShell module</a>, published the obnoxiously-named <a href="https://github.com/RamblingCookieMonster/PSLDAPQueryLogging">PSLDAPQueryLogging</a> module to GitHub and the PowerShell Gallery, and I’m ready for next time!</p> <p>Here’s a quick snippet where I enable logging on all domain controllers, pull when the logs, and disable logging:</p> <script src="https://gist.github.com/f4322853e049f2ec85c6.js"> </script> <h1 id="get-on-with-it">Get On With It!</h1> <p>Be sure to <a href="http://powershell.org/wp/2015/09/06/writing-and-publishing-powershell-modules/">share your modules</a> on GitHub and the <a href="https://www.powershellgallery.com/pages/GettingStarted">PowerShell Gallery</a>. Languages like <a href="https://www.perl.org/about/whitepapers/perl-cpan.html">Perl</a>, <a href="https://pypi.python.org/pypi">Python</a>, and <a href="https://rubygems.org/">Ruby</a> all have repositories like this, each with thousands and thousands of modules. We have 321. Let’s get to work!</p> <p>Barring the occasional <em>I have to share it!</em> discoveries, this will likely be my last post for some time; life is well-nigh to get crazy at home. In a good way. Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/Evil-LDAP-Queries/">Finding Evil LDAP Queries</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on October 02, 2015.</p> http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module 2016-07-24T05:30:00+00:00 2015-09-06T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#why-modules" id="markdown-toc-why-modules">Why Modules?</a></li> <li><a href="#this-seems-complicated" id="markdown-toc-this-seems-complicated">This Seems Complicated!</a></li> <li><a href="#the-ingredients" id="markdown-toc-the-ingredients">The Ingredients</a></li> <li><a href="#the-recipe" id="markdown-toc-the-recipe">The Recipe</a></li> <li><a href="#following-the-recipe" id="markdown-toc-following-the-recipe">Following the Recipe</a> <ul> <li><a href="#create-a-github-repository" id="markdown-toc-create-a-github-repository">Create a GitHub Repository.</a></li> <li><a href="#create-the-module-and-scaffolding-around-it" id="markdown-toc-create-the-module-and-scaffolding-around-it">Create the Module and ScaffoldingVirtuallyIt</a></li> <li><a href="#hook-up-appveyor-and-publish-the-module" id="markdown-toc-hook-up-appveyor-and-publish-the-module">Hook up AppVeyor and Publish the Module</a></li> </ul> </li> <li><a href="#how-i-write-modules-summarized" id="markdown-toc-how-i-write-modules-summarized">How I Write Modules, Summarized</a></li> <li><a href="#psstackexchange" id="markdown-toc-psstackexchange">PSStackExchange</a></li> <li><a href="#wrapping-up" id="markdown-toc-wrapping-up">Wrapping Up</a> <ul> <li><a href="#side-note-for-vendors" id="markdown-toc-side-note-for-vendors">Side Note for Vendors</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h3 id="rambling">Rambling</h3> <p>A while back, someone mentioned it might be fun to document the PowerShell-module-writing-process. This has been washed-up before, but I icon it would be fun to post my own process, from initial idea through publication on the official PowerShellGallery.com site.</p> <p>I recently discussed the <a href="http://powershell.org/wp/2015/08/30/2015-august-scripting-games-wrap-up/">August Scripting Games puzzle</a> on PowerShell.org, which involved querying a web API. It turns out this is a very worldwide need, and many of the modules we write utopian out these APIs into handy PowerShell functions and modules.</p> <p>We’re going to make the theorizing you know what a module is, and that you have some wits writing PowerShell functions. If not, be sure to spend some time <a href="http://ramblingcookiemonster.github.io/How-Do-I-Learn-PowerShell/">learning PowerShell</a> surpassing standing here!</p> <p>This post will imbricate my typical formula for writing a module, using the StackMartAPI as an example.Fingerself-ruling to <a href="https://github.com/RamblingCookieMonster/PSStackExchange/commit/db1277453374cb16684b35cf93a8f5c97288c41f">browse the PSStackExchange code</a> on your own.</p> <h3 id="why-modules">Why Modules?</h3> <p>Advanced functions will take you far with PowerShell. If you aren’t writing functions today, be sure to start encapsulating your lawmaking in these re-usable tools. But… they have their limits. Here are a few reasons you might stow your wide functions in a module:</p> <ul> <li>Simplify lawmaking organization</li> <li>Group related functions together</li> <li>Share state between functions, but not with the user</li> <li>Re-use “helper functions” that you don’t want exposed to the user</li> <li>Improve discoverability: <code class="highlighter-rouge">Find-Module MyModule</code> or <code class="highlighter-rouge">Get-Command -Module MyModule</code></li> <li>Simplify distribution: <code class="highlighter-rouge">Install-Module MyModule</code></li> </ul> <p>In our example, we will organize a set of StackMartfunctions into one module.</p> <h3 id="this-seems-complicated">This Seems Complicated!</h3> <p>Doing this from scratch might take you a little time. Thankfully, once you write a module or two, you can quickly get started by copying it and tweaking a few files. Don’t be scared off by the length of this post; writing your own modules is well worth spending a few minutes to pick up the basics!</p> <h3 id="the-ingredients">The Ingredients</h3> <p>There are many ways to create a module, from slapping a .psm1 extension onto a file, to compiling a fully fledged binary module <a href="http://www.powershellmagazine.com/2014/03/18/writing-a-powershell-module-in-c-part-1-the-basics/">from C#</a>. We’ll take a worldwide middle ground here, and use the pursuit ingredients:</p> <ul> <li><strong><a href="https://msdn.microsoft.com/en-us/library/dd878297">A Module Manifest</a></strong>. This is a .psd1 file that describes your module. <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/PSStackExchange.psd1">PSStackExchange.psd1</a></li> <li><strong>A Root Module</strong>. In our case, a script module .psm1 file. This is just PowerShell lawmaking to run when importing the module. <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/PSStackExchange.psm1">PSStackExchange.psm1</a></li> <li><strong>Exported (Public) Functions</strong>. These are the wide functions an end user can run from our module. For example, <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/Public/Get-SEQuestion.ps1">Get-SEQuestion.ps1</a> or <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/Public/Get-SEObject.ps1">Get-SEObject.ps1</a></li> <li><strong>Private Functions</strong>. These are optional “helper functions” that we want to use in our exported functions, that the end user shouldn’t see. For example, <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/Private/Add-ObjectDetail.ps1">Add-ObjectDetail.ps1</a> or <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/Private/Join-Parts.ps1">Join-Parts.ps1</a></li> <li><strong>Formats</strong>. These are optional format.ps1xml formats to help <a href="http://ramblingcookiemonster.github.io/Decorating-Objects/">decorate your output</a>, often specified in the module manifest ‘FormatsToProcess’. <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/PSStackExchange.Format.ps1xml">PSStackExchange.Format.ps1xml</a></li> <li><strong>Readme</strong>. If you’re <a href="http://ramblingcookiemonster.github.io/GitHub-For-PowerShell-Projects/">using GitHub</a> or flipside worldwide lawmaking repository, the Readme.md is a handy front page for your project, written using simple <a href="https://help.github.com/articles/github-flavored-markdown/">Markdown</a> rather than HTML</li> <li><strong>AppVeyor config</strong>. If you’re using a supported version tenancy solution, <a href="http://ramblingcookiemonster.github.io/GitHub-For-PowerShell-Projects/#continuous-integration">AppVeyor</a> enables simple and self-ruling continuous integration and wordage for unshut source projects. <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/appveyor.yml">AppVeyor.yml</a></li> </ul> <p>We have our ingredients, let’s squint at a recipe for a module!</p> <h3 id="the-recipe">The Recipe</h3> <p>We’re going to do this in a few quick steps:</p> <ul> <li>Create a GitHub repository</li> <li>Create the module and scaffolding virtually it</li> <li>Hook up AppVeyor and publish the module</li> </ul> <p>This might take a few minutes the first time you run through it, but you can infringe and tweak this same scaffolding for each module you write. In fact, you might find or write helper PowerShell modules and tools that simplify this process.</p> <p>Let’s get to work!</p> <h3 id="following-the-recipe">Following the Recipe</h3> <p>There’s no real order to this; depending on what you do or don’t incorporate, don’t finger like you need to follow this to the letter.</p> <h4 id="create-a-github-repository">Create a GitHub Repository.</h4> <p>This should be pretty straightforward. If you haven’t used GitHub before, the pursuit might help:</p> <ul> <li><a href="http://ramblingcookiemonster.github.io/GitHub-For-PowerShell-Projects">GitHub for PowerShell Projects</a></li> <li><a href="https://www.youtube.com/watch?v=wmPfDbsPeZY">PowerShell.org TechSession: A CrashUndertowin VersionTenancyand Git</a> and <a href="https://github.com/RamblingCookieMonster/Git-Presentation">presentation materials</a></li> </ul> <p>All we need to do is:</p> <ul> <li>Create an worth on GitHub, download GitHub for Windows</li> <li>Create a new repository (We’ll undeniability it PSStackExchange, and pick the MIT license)</li> <li>Clone PSStackExchange using GitHub for Windows</li> </ul> <p>Let’s move on to the most important bit, the module itself.</p> <h4 id="create-the-module-and-scaffolding-around-it">Create the Module and ScaffoldingVirtuallyIt</h4> <p>Here’s how I typically organize my modules. We’ll use PSStackExchange as an example, substitute this out for your own module!</p> <ul> <li><a href="https://github.com/RamblingCookieMonster/PSStackExchange/tree/db1277453374cb16684b35cf93a8f5c97288c41f">PSStackExchange\</a> <ul> <li>en-US\ (or locales of choice) <ul> <li>about_PSStackExchange.help.txt</li> </ul> </li> <li>Private<br /> <ul> <li>Join-Parts.ps1</li> <li>Get-SEData.ps1</li> <li>…</li> </ul> </li> <li>Public<br /> <ul> <li>Get-SEObject.ps1</li> <li>Search-SEQuestion.ps1</li> <li>…</li> </ul> </li> <li>lib\ (Not used in this module) <ul> <li>Some.Library.dll</li> </ul> </li> <li>bin\ (Not used in this module) <ul> <li>SomeDependency.exe</li> </ul> </li> <li>PSStackExchange.Format.ps1xml</li> <li>PSStackExchange.psd1</li> <li>PSStackExchange.psm1</li> </ul> </li> </ul> <p>If we’re going to be subtracting our project to GitHub or a similar lawmaking repository, we add a little increasingly scaffolding:</p> <ul> <li><a href="https://github.com/RamblingCookieMonster/PSStackExchange/tree/db1277453374cb16684b35cf93a8f5c97288c41f">Repository Root</a> <ul> <li>PSStackExchange\ (Module folder described above)</li> <li>Tests<br /> <ul> <li>PSStackExchange.Tests.ps1</li> <li>Appveyor.Pester.ps1</li> </ul> </li> <li>README.md</li> <li>AppVeyor.yml</li> </ul> </li> </ul> <p>I ran through the pursuit lawmaking to get started. Typically I’ll just reprinting the scaffolding from flipside module, create a new GUID in the psd1, and tweak other module specific references.</p> <script src="https://gist.github.com/12c4eb61dde8b2703184.js"> </script> <p>In our case, we have a few StackMartwide functions that hopefully follow <a href="http://ramblingcookiemonster.github.io/Building-PowerShell-Functions-Best-Practices/">a few weightier practices</a>, some private helper functions that we don’t want the user to see, and a few other files to imbricate testing and usability.</p> <p>In <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/PSStackExchange.psm1">PSStackExchange.psm1</a> we load our public and private functions. If a module is a work-in-progress, I’ll usually export $Public.Basename to stave nonflexible coding functions to export in the psd1. Once a module is released, I try to add the public functions to the psd1.</p> <p>If you’re writing a module, you should consider writing <a href="http://ramblingcookiemonster.github.io/GitHub-Pester-AppVeyor/#pester">Pester</a> tests for it. It’s quite comforting to have a suite of tests that run automatically without each transpiration you push, rather than thesping the lawmaking you write was correct, or attempting to manually test your lawmaking without each change.Requiteit a shot! We include a few superficial tests in <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/Tests/PSStackExchange.Tests.ps1">PSStackExchange.Tests.ps1</a>.</p> <p>Lastly, we include some usability features. We add an <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/en-US/about_PSStackExchange.help.txt">about_PSStackExchange</a> help topic, we <a href="http://ramblingcookiemonster.github.io/Decorating-Objects/">decorate our output</a> with the <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/PSStackExchange/PSStackExchange.Format.ps1xml">PSStackExchange.Format.ps1xml file</a>, and we add some notes on how to install and use the module in the <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/README.md">README.md</a>.</p> <p>We’re good to go! Let’s squint at how we can publish this module for others to use and improve.</p> <h4 id="hook-up-appveyor-and-publish-the-module">Hook up AppVeyor and Publish the Module</h4> <p>The content of our module is ready to publish.Surpassingwe publish this, we’ll enable continuous integration with some handy streamlined testing through AppVeyor.</p> <p>First, we <a href="http://ramblingcookiemonster.github.io/GitHub-Pester-AppVeyor/">set up our project in AppVeyor</a> by subtracting <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/master/appveyor.yml">appveyor.yml</a> to the repository, and subtracting the GitHub project to our AppVeyor account. We utopian out the calls to Pester in <a href="https://github.com/RamblingCookieMonster/PSStackExchange/blob/db1277453374cb16684b35cf93a8f5c97288c41f/Tests/appveyor.pester.ps1">AppVeyor.Pester.ps1</a>, using some <a href="http://ramblingcookiemonster.github.io/Github-Pester-AppVeyor-Part-2/">ideas from here</a>.</p> <p>Next, we push the changes we’ve made on our computer up to GitHub. Our lawmaking is now published, and AppVeyor will start <a href="https://ci.appveyor.com/project/RamblingCookieMonster/psstackexchange/build/1.0.3">running a build</a>.</p> <p>Lastly, we want to publish our module in the <a href="https://www.powershellgallery.com/">PowerShell Gallery</a>, giving end users with PowerShell 5 a simple way to find and install your module. We could vaccinate this up to automatically run in AppVeyor, but that’s <a href="http://ramblingcookiemonster.github.io/PSDeploy-Inception/">a topic for later</a>.</p> <ul> <li>Sign on to PowerShellGallery.com with your Microsoft account</li> <li>Get your API key (find it <a href="https://www.powershellgallery.com/account">here</a>)</li> <li>Publish your module!</li> </ul> <script src="https://gist.github.com/e7b381e6bf42c411765e.js"> </script> <p>Our module <a href="https://www.powershellgallery.com/packages/PSStackExchange/">is now live</a> on PowerShell Gallery!</p> <h3 id="how-i-write-modules-summarized">How I Write Modules, Summarized</h3> <p>Whew! That was a long post. Thankfully, most of this stuff can be re-used in each module you write. Let’s review the steps:</p> <ul> <li>Create a GitHub repository</li> <li>Create the module and scaffolding virtually it</li> <li>Hook up AppVeyor and publish the module</li> </ul> <p>The first and last step take a minute or two each. The module and scaffolding virtually it can be copied and tweaked, which should only take a few minutes. Most of your time will be spent writing the wide functions for the module.</p> <h3 id="psstackexchange">PSStackExchange</h3> <p>The module is published and ready to use! I’m on flipside computer with PowerShell 5, I can get up and running with a few lines of code:</p> <script src="https://gist.github.com/09f717821ce056a6dfe2.js"> </script> <p>Here’s some output from the examples:</p> <p><img src="/images/module/Get-SEObject.png" alt="Get-SEObject" /></p> <p><img src="/images/module/Search-SEQuestion.png" alt="Search-SEQuestion" /></p> <h3 id="wrapping-up">Wrapping Up</h3> <p>That’s well-nigh it! If you aren’t writing modules already, you should definitely consider it. Looking for remoter reading? Here are a few references that might come in handy:</p> <ul> <li><a href="https://github.com/RamblingCookieMonster/RamblingCookieMonster.github.io/blob/master/images/module/PSSummit2014-Freiheit-ModuleDesignRules.pptx?raw=true">ModuleDiamondRules</a> - This is from the 2014 PowerShell Summit, thanks to Kirk Freiheit</li> <li><a href="https://www.simple-talk.com/dotnet/.net-tools/further-down-the-rabbit-hole-powershell-modules-and-encapsulation/">FurtherLanugothe Rabbit Hole: PowerShell Modules and Encapsulation</a></li> <li><a href="https://msdn.microsoft.com/en-us/library/dd878297">How to Write a Module Manifest</a></li> <li><a href="https://msdn.microsoft.com/en-us/library/dd878324(v=vs.85).aspx">Windows PowerShell Modules</a></li> <li><a href="https://technet.microsoft.com/en-us/library/hh847804.aspx">about_Modules</a></li> <li><a href="https://ramblingcookiemonster.wordpress.com/2013/12/08/building-powershell-functions-best-practices/">Building PowerShell Functions -WeightierPractices</a> - Shameless plug. Includes a number of references.</li> <li><a href="https://www.manning.com/books/learn-powershell-toolmaking-in-a-month-of-lunches">Learn PowerShell Toolmaking in a Month of Lunches</a></li> <li><a href="http://mikefrobbins.com/2015/04/17/free-ebook-on-powershell-advanced-functions/">Free eBook on PowerShellWideFunctions</a></li> <li><a href="https://github.com/PoshCode/PowerShellPracticeAndStyle">The PowerShellWeightierPractices and Style Guide</a></li> <li><a href="http://www.jsnover.com/Docs/MonadManifesto.pdf">Monad Manifesto</a> - This gives a nice overview of the vision and goals set out for PowerShell. If you’re writing modules for public consumption, consider reading this, to stave publishing something as villainous as Citrix’ PVS “PowerShell” snapin.</li> </ul> <h4 id="side-note-for-vendors">Side Note for Vendors</h4> <p>Writing PSStackExchange <a href="http://ramblingcookiemonster.github.io/REST-PowerShell-and-Infoblox/">reminded me</a> how important it is for vendors of enterprise products to provide PowerShell modules that wrap their product’s API. Despite a nice API and decent documentation, writing a feature-poor PowerShell module for this was just as painful as <a href="http://ramblingcookiemonster.github.io/Querying-the-Infoblox-Web-API/">wrapping the Infoblox API</a>.</p> <p>Vendors: If your competition provides a PowerShell module and you do not, there’s a good endangerment I’ll push for your competitor’s product. This is a major value-add if you do it right and follow PowerShell conventions.</p> <p>Cheers!</p> <p><em>EDIT July 2016</em>: Updated links to PSStackExchange to <a href="https://twitter.com/psCookieMonster/status/757372331362779136">link to a specific point in time</a>. The current version of this project may see updates to illustrate things like PSDeploy.</p> <p><a href="http://ramblingcookiemonster.github.io/Building-A-PowerShell-Module/">Building a PowerShell Module</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on September 06, 2015.</p> http://ramblingcookiemonster.github.io/PowerShell-Configuration-Data 2015-08-16T22:00:00+00:00 2015-08-16T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#the-choices" id="markdown-toc-the-choices">The Choices</a></li> <li><a href="#xml" id="markdown-toc-xml">XML</a> <ul> <li><a href="#common-tools" id="markdown-toc-common-tools">Common tools</a></li> <li><a href="#the-good" id="markdown-toc-the-good">The Good</a></li> <li><a href="#the-bad" id="markdown-toc-the-bad">The Bad</a></li> <li><a href="#example" id="markdown-toc-example">Example</a></li> <li><a href="#should-i-use-xml" id="markdown-toc-should-i-use-xml">Should I use XML?</a></li> <li><a href="#further-reading" id="markdown-toc-further-reading">Further reading</a></li> </ul> </li> <li><a href="#json" id="markdown-toc-json">JSON</a> <ul> <li><a href="#common-tools-1" id="markdown-toc-common-tools-1">Common tools</a></li> <li><a href="#the-good-1" id="markdown-toc-the-good-1">The Good</a></li> <li><a href="#the-bad-1" id="markdown-toc-the-bad-1">The Bad</a></li> <li><a href="#example-1" id="markdown-toc-example-1">Example</a></li> <li><a href="#should-i-use-json" id="markdown-toc-should-i-use-json">Should I use JSON?</a></li> <li><a href="#further-reading-1" id="markdown-toc-further-reading-1">Further reading</a></li> </ul> </li> <li><a href="#yaml" id="markdown-toc-yaml">YAML</a> <ul> <li><a href="#common-tools-2" id="markdown-toc-common-tools-2">Common tools</a></li> <li><a href="#the-good-2" id="markdown-toc-the-good-2">The Good</a></li> <li><a href="#the-bad-2" id="markdown-toc-the-bad-2">The Bad</a></li> <li><a href="#example-2" id="markdown-toc-example-2">Example</a></li> <li><a href="#should-i-use-yaml" id="markdown-toc-should-i-use-yaml">Should I use YAML?</a></li> </ul> </li> <li><a href="#powershell-data-file-psd1" id="markdown-toc-powershell-data-file-psd1">PowerShell Data File (PSD1)</a> <ul> <li><a href="#common-tools-3" id="markdown-toc-common-tools-3">Common tools</a></li> <li><a href="#the-good-3" id="markdown-toc-the-good-3">The Good</a></li> <li><a href="#the-bad-3" id="markdown-toc-the-bad-3">The Bad</a></li> <li><a href="#example-3" id="markdown-toc-example-3">Example</a></li> <li><a href="#should-i-use-psd1" id="markdown-toc-should-i-use-psd1">Should I use PSD1?</a></li> </ul> </li> <li><a href="#other-data-formats" id="markdown-toc-other-data-formats">Other Data Formats</a></li> <li><a href="#what-should-i-use" id="markdown-toc-what-should-i-use">What Should I Use?</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h3 id="rambling">Rambling</h3> <p>I recently wrote <a href="https://ramblingcookiemonster.github.io/PSDeploy/">PSDeploy</a>, a quick-and-dirty module to utopian out PowerShell based deployments. You pinpoint what you want deployed in a configuration file, it does the rest.</p> <p>The most worldwide follow-up questions seemed to be <em>why yaml?</em> or <em>why not &#60;data format of preference&#62;?</em> This is a quick hit on the many data formats you can use from PowerShell.</p> <p><em><strong>Disclaimer</strong>: I know very little well-nigh data formats, their intended uses, or their benefits or caveats. This is from a layman’s perspective.</em></p> <h3 id="the-choices">The Choices</h3> <p>Let’s list off a few worldwide data formats we could use.</p> <ul> <li>XML</li> <li>JSON</li> <li>YAML</li> <li>PowerShell Data File (PSD1)</li> <li>INI</li> <li>Registry</li> <li>CSV</li> <li>Text</li> <li>Database</li> </ul> <p>Let’s take a peak at these from a upper level.Alimonyin mind these can vary wildly plane within a single format. For example, you might have an ad hoc JSON file, or a formal schema describing your JSON.</p> <p>For each solution you design, consider your needs, priorities, and constraints, and pair them up with one of these formats. Bear in mind the the lessons of the <a href="http://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html">configuration complexity clock</a>.</p> <h3 id="xml">XML</h3> <p>XML is a tried and true format, but is a bit inefficient and ugly.</p> <p><img src="/images/formats/xml.png" alt="XML example" /></p> <h4 id="common-tools">Common tools</h4> <ul> <li>Built in Cmdlets: Import-CliXml, Export-CliXml, ConvertTo-Xml</li> <li>Built in .NET framework support</li> </ul> <h4 id="the-good">The Good</h4> <ul> <li>Avoid external dependencies</li> <li>Flexible</li> <li>Wide cross-platform support and tooling</li> <li>CliXml functions provide simple serialization and deserialization</li> <li>Stores data with increasingly than one layer of depth</li> </ul> <h4 id="the-bad">The Bad</h4> <ul> <li>Not human readable</li> <li>Syntax is inefficient and verbose</li> <li>It’s 2015</li> </ul> <h4 id="example">Example</h4> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Serialize some data to disk</span> <span class="nb">Get-Item </span>C:\Windows\explorer.exe | <span class="nb">Export-Clixml</span> -Depth 5 -Path C:\XML.xml <span class="c1"># Deserialize the data</span> <span class="nv">$File</span> <span class="o">=</span> <span class="nb">Import-Clixml</span> -Path C:\XML.xml <span class="c1"># Drill down.</span> <span class="nv">$File</span>.VersionInfo.ProductVersion </code></pre></div></div> <h4 id="should-i-use-xml">Should I use XML?</h4> <p>There are two scenarios where I use XML. In all other cases I pick an alternative.</p> <ul> <li>Quick and dirty serialization. Import and Export CliXml are simple to use</li> <li>Technology lock-in. If it only supports XML, you don’t have a choice</li> </ul> <h4 id="further-reading">Further reading</h4> <ul> <li><a href="http://www.powershellmagazine.com/2013/08/19/mastering-everyday-xml-tasks-in-powershell/">Mastering everyday XML tasks in PowerShell</a></li> <li><a href="http://www.codeproject.com/Articles/61900/PowerShell-and-XML">PowerShell and XML</a></li> </ul> <h3 id="json">JSON</h3> <p>JSON is a lightweight data format worldwide in modern web APIs.</p> <p><img src="/images/formats/json.png" alt="JSON example" /></p> <h4 id="common-tools-1">Common tools</h4> <ul> <li>Built in Cmdlets: ConvertFrom-Json, ConvertTo-Json (PowerShell 3 or later)</li> </ul> <h4 id="the-good-1">The Good</h4> <ul> <li>Avoid external dependencies (in PowerShell 3 or later)</li> <li>Stores data with increasingly than one layer of depth</li> <li>Semi human readable</li> <li>Syntax is increasingly efficient and less verbose than XML</li> <li>Implemented in libraries wideness several languages</li> </ul> <h4 id="the-bad-1">The Bad</h4> <ul> <li>Want to store MSFT paths? Have fun: <code class="highlighter-rouge">{ "Path": "C:\\W\\T\\F" }</code></li> <li>Not as widespread cross-platform support or tooling as XML</li> </ul> <h4 id="example-1">Example</h4> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Convert some data to Json</span> <span class="nv">$JSON</span> <span class="o">=</span> <span class="nb">Get-Item </span>C:\Windows\explorer.exe | <span class="nb">ConvertTo-Json</span> -Depth 2 <span class="c1"># Read the Json when into an object</span> <span class="nv">$File</span> <span class="o">=</span> <span class="nv">$JSON</span> | <span class="nb">ConvertFrom-Json</span> <span class="c1"># Drill down.</span> <span class="nv">$File</span>.VersionInfo.ProductVersion </code></pre></div></div> <h4 id="should-i-use-json">Should I use JSON?</h4> <p>JSON is a popular and unscratched nomination nowadays. Depending on your needs, this is often a good fit.</p> <h4 id="further-reading-1">Further reading</h4> <ul> <li><a href="http://www.powershellmagazine.com/2014/12/01/a-json-primer-for-administrators/">A JSON Primer for Administrators</a></li> <li><a href="https://github.com/JamesNK/Newtonsoft.Json">JSON.NET</a></li> <li><a href="http://blogs.technet.com/b/heyscriptingguy/archive/2014/04/23/json-is-the-new-xml.aspx">JSON Is the New XML</a></li> </ul> <h3 id="yaml">YAML</h3> <p>YAML is a human friendly data format.</p> <p><img src="/images/formats/yaml.png" alt="yaml example" /></p> <h4 id="common-tools-2">Common tools</h4> <ul> <li><a href="https://github.com/scottmuc/PowerYaml">PowerYaml</a> and various <a href="https://github.com/cdhunt/PowerYaml">forks</a></li> <li><a href="https://github.com/aaubry/YamlDotNet">Yaml.Net</a></li> </ul> <h4 id="the-good-2">The Good</h4> <ul> <li>Very human readable</li> <li>Stores data with increasingly than one layer of depth</li> <li>Syntax is increasingly efficient and less verbose than XML</li> <li>Implemented in libraries wideness several languages</li> </ul> <h4 id="the-bad-2">The Bad</h4> <ul> <li>Dependencies on libraries like Yaml.Net</li> <li>Haven’t seen a reliable serialize and deserialize module for YAML yet</li> <li>Whitespace is part of the syntax</li> </ul> <h4 id="example-2">Example</h4> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#Pinpointsome Yaml</span> <span class="nv">$Yaml</span> <span class="o">=</span> @<span class="s2">" Name: explorer.exe Length: 4532304 DirectoryName: C:\Windows VersionInfo: ProductVersion: 10.0.10240.16384 "</span>@ <span class="c1">#Read the Yaml, using PowerYaml</span> <span class="nv">$File</span> <span class="o">=</span> Get-Yaml -FromString <span class="nv">$Yaml</span> <span class="c1"># Drill down.</span> <span class="nv">$File</span>.VersionInfo.ProductVersion </code></pre></div></div> <h4 id="should-i-use-yaml">Should I use YAML?</h4> <p>I would only recommend YAML where human readability is paramount, and your other needs and constraints don’t rule it out.</p> <p>Yaml a unconfined option when you will simply be reading in a config file, and the end users will be manually manipulating this file.</p> <h3 id="powershell-data-file-psd1">PowerShell Data File (PSD1)</h3> <p>PowerShell data files are used for PowerShell module manifests, but can be used to store wrong-headed data.</p> <p><img src="/images/formats/psd1.png" alt="psd1 example" /></p> <h4 id="common-tools-3">Common tools</h4> <ul> <li>Built in Cmdlet: <a href="https://technet.microsoft.com/en-us/library/hh849919.aspx">Import-LocalizedData</a></li> </ul> <h4 id="the-good-3">The Good</h4> <ul> <li>Familiar to PowerShell authors</li> <li>Avoid external dependencies</li> <li>Stores data with increasingly than one layer of depth</li> <li>Semi human readable</li> </ul> <h4 id="the-bad-3">The Bad</h4> <ul> <li>Single platform</li> <li>Haven’t seen a reliable serialize and deserialize module for PSD1 files yet</li> </ul> <h4 id="example-3">Example</h4> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Create a PSD1 file</span> @<span class="s2">" @{ Name = 'explorer.exe' Length = 4532304 DirectoryName = 'C:\Windows' VersionInfo = @{ ProductVersion = '10.0.10240.16384' } } "</span>@ | <span class="nb">Out-File</span> -FilePath C:\PSD1.psd1 <span class="c1"># Read the file</span> <span class="nv">$File</span> <span class="o">=</span> <span class="nb">Import-LocalizedData</span> -BaseDirectory C:\ -FileName PSD1.psd1 <span class="c1"># Drill down.</span> <span class="nv">$File</span>.VersionInfo.ProductVersion </code></pre></div></div> <h4 id="should-i-use-psd1">Should I use PSD1?</h4> <p>This seems like a reasonable nomination for PowerShell configuration files that will be edited by hand, by folks familiar with PowerShell.</p> <h3 id="other-data-formats">Other Data Formats</h3> <p>There are a variety of other choices. Here are a few others you might consider:</p> <ul> <li><strong>CSV</strong> has plenty of built in Cmdlets, but is quite limited and might produce <a href="http://learn-powershell.net/2014/01/24/avoiding-system-object-or-similar-output-when-using-export-csv/">unexpected results</a>.</li> <li><strong>Text</strong> is an option, with a variety of tools including the new <a href="http://www.powershellmagazine.com/2014/09/09/using-the-convertfrom-string-cmdlet-to-parse-structured-text/">ConvertFrom-String</a>. Not sure why you would chose this over an existing data format.</li> <li><strong>The registry</strong>. I wouldn’t pick it, but it’s used by many applications and is familiar to most Windows administrators. Tools like <a href="https://psremoteregistry.codeplex.com/releases/view/65928">PSRemoteRegistry</a> make this simple to work with remotely, unlike the registry PSProvider.</li> <li><strong><a href="http://lipkau.github.io/PsIni/">Ini files</a></strong> are a bit dated, but are simple to read and use. Given their limitations and the wealth of other options, you should probably leave these in the attic.</li> <li><strong>Databases</strong> are a unconfined option for larger solutions, or where your data model requires a bit increasingly sophistication. <a href="http://ramblingcookiemonster.github.io/SQLite-and-PowerShell/">SQLite</a> is a handy, cross-platform solution, if SQL Server or other database engines are too heavy-weight.</li> </ul> <h3 id="what-should-i-use">What Should I Use?</h3> <p>Like most answers in the world of computing: It depends. Chances are, plane with your particular needs and constraints, there isn’t one correct choice. Consider your options, and pick a data format that makes sense to you.</p> <p>Do you enjoy trolling? Consider JSONx:</p> <p><a href="https://twitter.com/DanHarper7/status/514822464673951744"><img src="/images/formats/jsonx.png" alt="JSONx tweet" /></a></p> <p>Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/PowerShell-Configuration-Data/">PowerShell Configuration Data</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on August 16, 2015.</p> http://ramblingcookiemonster.github.io/PSDeploy 2015-08-15T22:00:00+00:00 2015-08-15T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#continuous-deployment" id="markdown-toc-continuous-deployment">Continuous Deployment</a></li> <li><a href="#psdeploy" id="markdown-toc-psdeploy">PSDeploy</a> <ul> <li><a href="#ingredients" id="markdown-toc-ingredients">Ingredients</a></li> <li><a href="#a-simple-illustration" id="markdown-toc-a-simple-illustration">A Simple Illustration</a></li> <li><a href="#getting-started" id="markdown-toc-getting-started">Getting Started</a></li> </ul> </li> <li><a href="#example-deployment-scenarios" id="markdown-toc-example-deployment-scenarios">Example Deployment Scenarios</a> <ul> <li><a href="#basic-filesystem-deployment" id="markdown-toc-basic-filesystem-deployment">Basic Filesystem Deployment</a></li> <li><a href="#filesystemremote-deployment" id="markdown-toc-filesystemremote-deployment">FilesystemRemote Deployment</a></li> </ul> </li> <li><a href="#extending-psdeploy" id="markdown-toc-extending-psdeploy">Extending PSDeploy</a> <ul> <li><a href="#update-psdeployyml" id="markdown-toc-update-psdeployyml">Update PSDeploy.yml</a></li> <li><a href="#create-the-deployment-script" id="markdown-toc-create-the-deployment-script">Create the Deployment Script</a></li> <li><a href="#yaml-options" id="markdown-toc-yaml-options">YAML Options</a></li> </ul> </li> <li><a href="#next-steps" id="markdown-toc-next-steps">Next Steps</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h3 id="rambling">Rambling</h3> <p>A short while when I gave my first live <a href="https://github.com/RamblingCookieMonster/Git-Presentation">webinar</a>, on getting started with Git and GitHub. It was both heady and a bit terrifying. I’m gearing up for similar sessions at work to get folks going with our version tenancy solution, Stash.</p> <p>When I squint at version tenancy as an IT professional, one pain point that we could help with stands out. In wing to waffly their workflow and learning version control, folks are now working with files and folders in source control. Not where they live. Wouldn’t it be handy if we could use version control, and not worry well-nigh remembering to be sure to move the files to their towardly homes?</p> <p>This post will discuss a simple PowerShell based deployment solution: <a href="https://github.com/RamblingCookieMonster/PSDeploy">PSDeploy</a>.</p> <h3 id="continuous-deployment">Continuous Deployment</h3> <p>Developers realized this was important long ago. There are some specialized deployment tools like <a href="http://octopusdeploy.com/">Octopus Deploy</a>, continuous integration and deployment tools like <a href="http://www.appveyor.com/">AppVeyor</a> or <a href="https://www.jetbrains.com/teamcity/">TeamCity</a>, and continuous integration tools like <a href="http://jenkins-ci.org/">Jenkins</a> that can be shoehorned into providing deployments.</p> <p>So, with all these solutions, why duct-tape something together?</p> <ul> <li>I like fun side projects</li> <li>A simplified deployment system might help with buy in for our version tenancy solution</li> <li>We use <a href="https://www.hodgkins.net.au/powershell/automating-with-jenkins-and-powershell-on-windows-part-1/">Jenkins</a>. Their build definition process leaves a bit to be desired. This allows me to use the same visionary build script for every project, as long as I have a deployments.yml in the repo root</li> <li>This is portable, and might help folks who use increasingly than one tool chain. I could use PSDeploy with Jenkins, TeamCity, AppVeyor, Bamboo, etc.</li> </ul> <p>Let’s swoop in.</p> <h3 id="psdeploy">PSDeploy</h3> <p>This is a quick and dirty module that simplifies deployments. You create a deployment config file, PSDeploy reads it and runs your deployment(s).</p> <h4 id="ingredients">Ingredients</h4> <ul> <li><strong>YAML</strong>: <a href="https://en.wikipedia.org/wiki/YAML">Data format</a> for deployment config files.Planeeasier to read than JSON. Familiar to folks using tools like AppVeyor.</li> <li><strong>Deployment config</strong>: YAML files defining what is stuff deployed. They should have a source, a destination, a deployment type, and might contain freeform deployment options.</li> <li><strong>Deployment type</strong>: These pinpoint how to unquestionably deploy something. Each type is associated with a script. We’re including FileSystem and FileSystemRemote to start, but this is extensible.</li> <li><strong>Deployment script</strong>: These are scripts associated with a particular deployment type. All should winnow a ‘Deployment’ parameter. For example, the FileSystem script uses robocopy and copy-item to deploy folders and files, respectively.</li> </ul> <h4 id="a-simple-illustration">A Simple Illustration</h4> <p>Let’s looks at an example deployments.yml and the outcome of invoking it.</p> <p>Here’s a project I’m working on. I want to deploy MyModule to a few paths every time I push a commit. I should probably gate this with Pester tests as well : )</p> <p><img src="/images/psdeploy/module.png" alt="Module folder" /></p> <p>Here are the content of the deployments.yml:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">MyModuleDeployment</span><span class="pi">:</span> <span class="na">Author</span><span class="pi">:</span> <span class="s1">'</span><span class="s">wframe'</span> <span class="na">Source</span><span class="pi">:</span> <span class="pi">-</span> <span class="s1">'</span><span class="s">MyModule'</span> <span class="na">Destination</span><span class="pi">:</span> <span class="pi">-</span> <span class="s1">'</span><span class="s">C:\Temp\MyModule'</span> <span class="pi">-</span> <span class="s1">'</span><span class="s">\\C-IS-TS-91\C$\Users\wframem\documents\WindowsPowerShell\Modules\MyModule'</span> <span class="na">DeploymentType</span><span class="pi">:</span> <span class="s">Filesystem</span> <span class="na">Options</span><span class="pi">:</span> <span class="na">Mirror</span><span class="pi">:</span> <span class="s">True</span> </code></pre></div></div> <p>We can verify how PSDeploy will parse this:</p> <p><img src="/images/psdeploy/get-psdeployment.png" alt="Get-PSDeployment" /></p> <p>Let’s invoke the deployment:</p> <p><img src="/images/psdeploy/mymoduledeployment.png" alt="Invoke-PSDeployment" /></p> <p>If we trammels the destination paths, we find MyModule has been deployed!</p> <p><img src="/images/psdeploy/deployed.png" alt="Deployed" /></p> <p>What happened under the hood?</p> <ul> <li>We read the deployments.yml</li> <li>We see that the source is a folder</li> <li>We see that the deployment type is Filesystem</li> <li>We squint up the script for filesystem deployments</li> <li>We execute the filesystem script for this deployment</li> <li>The script reads the deployment and processes it: <ul> <li>It’s a folder, so we use robocopy /E /XO</li> <li>The ‘mirror’ deployment option is set, so we add on /PURGE</li> </ul> </li> </ul> <p>If you prefer illustrations, here’s quick diagram of a similar deployment:</p> <p><a href="https://cloud.githubusercontent.com/assets/6377597/9177949/7c5a0c26-3f62-11e5-9d31-61f74a324383.png"><img src="https://cloud.githubusercontent.com/assets/6377597/9177951/7fec1fa0-3f62-11e5-98bc-6a077d1c57f0.png" alt="psdeployflowsmall" /></a></p> <p>That’s pretty much it! Instead of defining all this lawmaking in my continuous deployment system, I just run Invoke-PSDeployment.</p> <h4 id="getting-started">Getting Started</h4> <p>First off, download and explore the module:</p> <script src="https://gist.github.com/07701d9e29d55593962a.js"> </script> <p>Running through the lawmaking above, we see a few commands you can work with:</p> <ul> <li><strong>Get-PSDeployment</strong>: Read a deployment config file</li> <li><strong>Get-PSDeploymentType</strong>: Get details on a deployment type</li> <li><strong>Get-PSDeploymentScript</strong>: Show misogynist deployment types and associated scripts</li> <li><strong>Invoke-PSDeployment</strong>: Run a deployment</li> </ul> <p>Let’s put this to work in a few real world scenarios.</p> <h3 id="example-deployment-scenarios">Example Deployment Scenarios</h3> <h4 id="basic-filesystem-deployment">Basic Filesystem Deployment</h4> <p>I want to deploy C:\Git\MyModuleRepo\MyModule to \\Server\Modules. How can I use PSDeploy to do this?</p> <p>First, we add a config file, C:\Git\MyModuleRepo\deployments.yml. Here’s the content:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">MyModuleDeployment</span><span class="pi">:</span> <span class="na">Author</span><span class="pi">:</span> <span class="s1">'</span><span class="s">wframe'</span> <span class="na">Source</span><span class="pi">:</span> <span class="pi">-</span> <span class="s1">'</span><span class="s">MyModule'</span> <span class="na">Destination</span><span class="pi">:</span> <span class="pi">-</span> <span class="s1">'</span><span class="s">\\Server\Modules\MyModule'</span> <span class="na">DeploymentType</span><span class="pi">:</span> <span class="s">Filesystem</span> <span class="na">Options</span><span class="pi">:</span> <span class="na">Mirror</span><span class="pi">:</span> <span class="s">True</span> </code></pre></div></div> <p>We verify that the deployment will do what we want:</p> <script src="https://gist.github.com/641149d25ad3cfa80529.js"> </script> <p>Looks good to me! In our continuous deployment solution, we simply run this:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code> Invoke-PSDeployment -Path <span class="s1">'C:\Git\MyModuleRepo\deployments.yml'</span> </code></pre></div></div> <p><a href="https://gist.github.com/RamblingCookieMonster/bff0d1a4604ba7a7fd23">This gist</a> shows how you can test out PSDeploy without setting up any fancy tools. All you do is create a dummy folder to deploy, targets to deploy it to, and invoke the deployment.</p> <p>Let’s squint at a increasingly involved deployment.</p> <h4 id="filesystemremote-deployment">FilesystemRemote Deployment</h4> <p>So! I’m using Jenkins for continuous integration and trying to shoehorn in a deployment. I’m running into trouble: Jenkins is running with an worth that doesn’t have the right privileges to deploy to the targets.</p> <p>We can use the FilesystemRemote deployment type, which will run the robocopy and copy-item commands in a PowerShell remoting session with towardly credentials. I know, it’s silly, pull requests for a increasingly towardly solution would be welcome : )</p> <p>Ahead of time, I locked lanugo the Jenkins server and set up EnvInject, with a little help from <a href="https://www.hodgkins.net.au/powershell/automating-with-jenkins-and-powershell-on-windows-part-2/">Matt Hodgkins’ post</a>.</p> <p>Next, I set up a project with the pursuit PowerShell build command. I’m omitting the testing steps for simplicity. You should strongly consider gating your deployments using Pester tests.</p> <script src="https://gist.github.com/dd110b56466fd918f672.js"> </script> <p>This looks a bit complicated, but all we’re doing is loading up PSDeploy, grabbing credentials for our deployment, and kicking off the deployment with some parameters. You can run <code class="highlighter-rouge">Get-Help Invoke-PSDeployment -Parameter DeploymentParameters</code> for increasingly information on passing parameters to a deployment script.</p> <p>That’s well-nigh it! Maybe SomeSessionConfig is a delegated endpoint, which works virtually the double hop issue. I could moreover configure a <a href="http://www.powershellmagazine.com/2014/03/06/accidental-sabotage-beware-of-credssp/">CredSSP</a> endpoint, but that might yank the wrath of the security community. Tip: Do consider the risks of CredSSP, but if you still have admins using RDP, you’re once using CredSSP, and you have worthier problems to worry well-nigh : )</p> <p>We just walked through a few example deployment types. What if you want increasingly deployment options?</p> <h3 id="extending-psdeploy">Extending PSDeploy</h3> <p>I’m probably getting superiority of myself, but I tried to make PSDeploy somewhat extensible. Follow these steps to add a new deployment type:</p> <ul> <li>Update PSDeploy.yml (Associates scripts to deployment types)</li> <li>Write the deployment script</li> <li>Consider whether and where to store options in the deployment config</li> </ul> <h4 id="update-psdeployyml">Update PSDeploy.yml</h4> <p>This file is stored in the root of the module, although you can move it to a inside spot. It stores the associations of deployment types to deployment scripts.</p> <ul> <li>The deployment type is the root node.</li> <li>The script node defines what script to run for these deployment types</li> <li>The unravelment is exposed when running Get-PSDeploymentType</li> </ul> <p>For example, I might add support for SCP deployments:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">SCP</span><span class="pi">:</span> <span class="na">Script</span><span class="pi">:</span> <span class="s">SCP.ps1</span> <span class="na">Description</span><span class="pi">:</span> <span class="s">Deploys artifacts using SCP. Requires Posh-SSH</span> </code></pre></div></div> <h4 id="create-the-deployment-script">Create the Deployment Script</h4> <p>These are stored in the PSDeploy folder under PSDeployScripts, although you can point to other paths in PSDeploy.yml</p> <ul> <li>Following the SCP example, our deployment script might be C:\Path\To\PSDeploy\PSDeployScripts\SCP.ps1</li> <li>In your deployment script (SCP.ps1), include a ‘Deployment’ parameter <ul> <li>See <a href="https://github.com/RamblingCookieMonster/PSDeploy/blob/master/PSDeploy/PSDeployScripts/FilesystemRemote.ps1">FilesystemRemote.ps1</a> for an example</li> </ul> </li> <li>Include scuttlebutt based help to explain the deployment and any uneaten parameters you accept. This is a good spot to describe YAML options, if you use them.</li> </ul> <p>Here’s how I implement the deployment parameter in the filesystem deployment scripts:</p> <script src="https://gist.github.com/f5cdb592f2bae3039960.js"> </script> <h4 id="yaml-options">YAML Options</h4> <p>If you want to include options in a deployment config file, similar to the mirror option we use in Filesystem deployments, just include them in deployments.yml.</p> <ul> <li>Get-PSDeployment converts the deployment config YAML and processes it into a number of ‘deployment’ objects that are passed to your script.</li> <li>No special steps are needed for the deployment config, just add nodes as needed. <ul> <li>We pericope the YAML Options node with Get-PSDeployment, and process it into a DeploymentOptions property on the Deployment object</li> <li>If you need plane increasingly flexibility, we store the raw converted YAML in the Raw property returned from Get-PSDeployment</li> </ul> </li> </ul> <p>Here’s an example extended deployment config with a hostname and port option:</p> <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">SomeExtendedDeployment</span><span class="pi">:</span> <span class="na">Author</span><span class="pi">:</span> <span class="s1">'</span><span class="s">wframe'</span> <span class="na">Source</span><span class="pi">:</span> <span class="pi">-</span> <span class="s1">'</span><span class="s">SomeSourceFolder'</span> <span class="na">Destination</span><span class="pi">:</span> <span class="pi">-</span> <span class="s1">'</span><span class="s">/Some/Target/Path'</span> <span class="na">DeploymentType</span><span class="pi">:</span> <span class="s">SCP</span> <span class="na">Options</span><span class="pi">:</span> <span class="na">Hostname</span><span class="pi">:</span> <span class="s">Server1</span> <span class="na">Port</span><span class="pi">:</span> <span class="s">22</span> </code></pre></div></div> <p>We can explore this with Get-PSDeployment to see what we would work with in out deployment script:</p> <p><img src="/images/psdeploy/deploymentyml.png" alt="DeploymentOptions" /></p> <h3 id="next-steps">Next Steps</h3> <p>That’s pretty much it! I’m hoping that we get increasingly IT professionals using version control, continuous integration, continuous delivery, and similar solutions. We might see increasingly (and better!) tools to simplify and streamline using these solutions as an IT professional.</p> <p>Don’t be wrung of tools and ideas traditionally associated with developers or ‘DevOps’.Planeif you work in an enterprise that has few if any developers, these tools and the ideas overdue them can goody IT as a whole.</p> <p>If you haven’t already, you should consider <a href="http://stevenmurawski.com/powershell/2015/8/moving-in-to-open-source">joining the unshut source community</a>. Suggestions, pull requests, and other contributions to PSDeploy would be increasingly than welcome!</p> <p><a href="http://ramblingcookiemonster.github.io/PSDeploy/">PSDeploy&#58; Simplified PowerShell Based Deployments</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on August 15, 2015.</p> http://ramblingcookiemonster.github.io/RabbitMQ-Intro 2015-07-06T22:00:00+00:00 2015-07-06T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#rabbitmq" id="markdown-toc-rabbitmq">RabbitMQ</a></li> <li><a href="#basic-installation" id="markdown-toc-basic-installation">Basic installation</a> <ul> <li><a href="#troubleshooting" id="markdown-toc-troubleshooting">Troubleshooting</a></li> </ul> </li> <li><a href="#managing-rabbitmq-with-rabbitmqtools" id="markdown-toc-managing-rabbitmq-with-rabbitmqtools">Managing RabbitMQ with RabbitMQTools</a></li> <li><a href="#sending-and-receiving-messages-with-psrabbitmq" id="markdown-toc-sending-and-receiving-messages-with-psrabbitmq">Sending and receiving messages with PSRabbitMQ</a></li> <li><a href="#wrap" id="markdown-toc-wrap">Wrap</a> <ul> <li><a href="#rambling-aside" id="markdown-toc-rambling-aside">Rambling aside</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h3 id="rambling">Rambling</h3> <p>I’m finally transmissible up on some of the ideas and takeaways from the <a href="http://ramblingcookiemonster.github.io/PowerShell-Summit-Wrap/">PowerShell Summit</a>. One of the fun <em>oh, I have to try that!</em> ideas came up when <a href="https://twitter.com/gpduck">Chris Duck</a> and <a href="https://twitter.com/rjasonmorgan">Jason Morgan</a> started chatting well-nigh something tabbed <a href="https://www.rabbitmq.com/getstarted.html">RabbitMQ</a>, a messaging solution.</p> <p>The timing worked out perfectly. I’m running into increasingly and increasingly processes where tooling or automation would require some form of orchestration. We’ll likely go with a COTS product at some point, but the idea of locking myself into a specific vendor’s specific version of an orchestration solution is not something I’m going to rush into.</p> <p>Let’s run through a quick overview of RabbitMQ, some notes on a vital install, and some PowerShell tools you could use to help manage and use RabbitMQ in your own solutions.</p> <p><strong>Disclaimer</strong>: <em>This was my first foray into the world of messaging, I am by no ways an expert</em></p> <h3 id="rabbitmq">RabbitMQ</h3> <p>Paraphrasing their <a href="https://www.rabbitmq.com/">front page</a>, RabbitMQ is a robust, easy to use, navigate platform, unshut source solution for messaging.</p> <p>Messaging… isn’t that something for developers? I work on the systems side of things, why would I need that? It turns out messaging can be quite helpful in the world of IT professionals.</p> <ul> <li>Avoid fragile, monolithic scripts</li> <li>Share data between scripts running on various systems and platforms</li> <li>Avoid complex, non-standard “messaging” - Oh, my script watches for a file / Windows event / SQL data change, etc.</li> <li>Deliver messages reliably</li> <li>Perform tasks in order</li> <li>Buffer things up, process at your leisure</li> </ul> <p>References on messaging abound, from the <a href="https://www.youtube.com/watch?v=ZQogoEVXBSA">cheeky</a> (<a href="http://www.rabbitmq.com/resources/google-tech-talk-final/alexis-google-rabbitmq-talk.pdf">pres materials</a>), to <a href="https://www.rabbitmq.com/tutorials/amqp-concepts.html">high level conceptual bits</a>, to <a href="https://www.rabbitmq.com/getstarted.html">practical RabbitMq tutorials</a>.</p> <p>A worldwide illustration on the basics, apologies if I butcher this:</p> <ul> <li>Your various scripts (<em>publishers</em>) are sending mail (<em>messages</em>)</li> <li>The mail might have an write (<em>routing key</em>) to help route it to the right recipient</li> <li>The mail gets dropped in a USPS mailbox (an <em>exchange</em>)</li> <li>The mail is routed (<em>binding</em>) to the towardly recipient mailboxes (<em>queues</em>)</li> <li>The recipients (<em>consumers</em>) might stop by a PO box to pick up their mail, or might have it delivered (<em>subscriptions</em>)</li> </ul> <p><a href="https://www.rabbitmq.com/tutorials/amqp-concepts.html"><img src="/images/rabbitmq/hello-world-example-routing.png" alt="Basic illustration" /></a></p> <p>Finally, here’s a simple example showing two self-sustaining sessions talk to each other through PSRabbitMQ:</p> <p><img src="/images/rabbitmq/Listener.gif" alt="Listener gif" /></p> <p>Let’s squint at setting up a single RabbitMQ server.</p> <h3 id="basic-installation">Basic installation</h3> <p>If you’re planning to use this for increasingly than a quick POC, <a href="http://www.rabbitmq.com/admin-guide.html">read up</a> and configure per your own needs and requirements.</p> <p>Some rough notes taken during the single server POC deployment we stood up:</p> <ul> <li>Specify a wiring path, unless you want RabbitMQ running out of AppData</li> </ul> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$rabbitdir</span> <span class="o">=</span> <span class="s1">'C:\RabbitMQ'</span> mkdir <span class="nv">$rabbitdir</span> mkdir <span class="nv">$rabbitdir</span>\ssl <span class="o">[</span>Environment]::SetEnvironmentVariable<span class="o">(</span><span class="s2">"RABBITMQ_BASE"</span>, <span class="nv">$rabbitdir</span>, <span class="s2">"Machine"</span><span class="o">)</span> </code></pre></div></div> <ul> <li><a href="http://www.erlang.org/download.html">Download and install Erlang</a> - I went with x64</li> <li><a href="http://www.rabbitmq.com/install-windows.html">Install RabbitMQ</a></li> <li>If you’re planning to use SSL, grab the latest OpenSSL (variety of <a href="http://indy.fulgan.com/SSL/">sources</a>), waif the files in C:\RabbitMQ\ssl</li> <li>Enable the web interface and RESTful API</li> </ul> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#Transpirationthis out depending on your RabbitMQ install location:</span> <span class="nv">$sbin</span> <span class="o">=</span> <span class="s2">"C:\Program Files (x86)\RabbitMQ Server\rabbitmq_server-3.5.3\sbin"</span> &amp; <span class="nv">$sbin</span>\rabbitmq-plugins.bat <span class="nb">enable </span>rabbitmq_management <span class="c1">#commit changes be re-installing service</span> &amp; <span class="nv">$Sbin</span>\rabbitmq-service.bat stop &amp; <span class="nv">$Sbin</span>\rabbitmq-service.bat remove &amp; <span class="nv">$Sbin</span>\rabbitmq-service.bat install &amp; <span class="nv">$Sbin</span>\rabbitmq-service.bat <span class="nb">start</span> </code></pre></div></div> <p>At this point, you should be worldly-wise to scan to http://localhost:15672, but we aren’t washed-up yet!</p> <ul> <li>Set up SSL. Some of this might be redundant</li> </ul> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#UnshutPowerShell.exe - this won't work in the ISE</span> <span class="c1"># I copied my certs here temporarily...</span> <span class="c1"># I have my domain's wildcard cert in a contoso.org.pfx file and CA public certs in public.cer and publitROOT.cer</span> <span class="nv">$rabbitdir</span> <span class="o">=</span> <span class="s1">'C:\RabbitMQ'</span> <span class="nb">cd</span> <span class="nv">$rabbitdir</span>\ssl <span class="c1">#Certs for your CA</span> .\openssl x509 -inform der -in public.cer -out public.pem .\openssl x509 -inform der -in publicROOT.cer -out publicROOT.pem <span class="c1"># Add the contents of publicROOT.pem to public.pem</span> <span class="c1"># Use PowerShell, notepad2, or something else that won't mess with encoding.</span> .\openssl pkcs12 -in contoso.org.pfx -out server.key -nocerts -nodes <span class="c1">#pw prompt here</span> .\openssl rsa -in server.key -out rsa.server.key .\openssl pkcs12 -in contoso.org.pfx -out server.pem -nokeys <span class="c1">#pw prompt here</span> </code></pre></div></div> <ul> <li>Create a <a href="https://gist.github.com/RamblingCookieMonster/d0ca18ca59ee11082bb8">rabbitmq.config</a> file in $env:RABBITMQ_BASE that we set earlier.RetreadSSL options as needed.</li> <li>Re-install the service one increasingly time…</li> </ul> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#commit changes be re-installing service</span> &amp; <span class="nv">$Sbin</span>\rabbitmq-service.bat stop &amp; <span class="nv">$Sbin</span>\rabbitmq-service.bat remove &amp; <span class="nv">$Sbin</span>\rabbitmq-service.bat install &amp; <span class="nv">$Sbin</span>\rabbitmq-service.bat <span class="nb">start</span> </code></pre></div></div> <ul> <li>Configure accounts. Be sure to remove the default guest account!</li> </ul> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#Add users and passwords. This admin worth has wangle to everything...</span> &amp; <span class="nv">$Sbin</span>\rabbitmqctl.bat add_user zookeeper <span class="s2">"SUPERSECUREPASSWORD!"</span> &amp; <span class="nv">$Sbin</span>\rabbitmqctl.bat set_permissions zookeeper <span class="s2">".*"</span> <span class="s2">".*"</span> <span class="s2">".*"</span> &amp; <span class="nv">$Sbin</span>\rabbitmqctl.bat set_user_tags zookeeper zookeeper <span class="c1">#Example subtracting my self with wangle to all queues, and as an administrator</span> &amp; <span class="nv">$Sbin</span>\rabbitmqctl.bat add_user cmonster <span class="s2">"SUPERSECUREPASSWORD!"</span> &amp; <span class="nv">$Sbin</span>\rabbitmqctl.bat set_permissions cmonster <span class="s2">".*"</span> <span class="s2">".*"</span> <span class="s2">".*"</span> &amp; <span class="nv">$Sbin</span>\rabbitmqctl.bat set_user_tags cmonster zookeeper <span class="c1">#The permissions section is a regex for what queues you have wangle to, CONFIGURE, WRITE, READ. I have .* (regex for EVERYTHING!) for each, meaning I can config, write, and read anything</span> <span class="c1">#https://www.rabbitmq.com/access-control.html</span> <span class="c1">#https://www.rabbitmq.com/man/rabbitmqctl.1.man.html#Access%20control</span> <span class="c1">#Delete the guest account, it has full admin and is evil.</span> &amp; <span class="nv">$Sbin</span>\rabbitmqctl.bat delete_user guest </code></pre></div></div> <p>Hopefully everything worked and you can now scan to https://servername.contoso.org:15671! If you skipped the SSL and rabbitmq.config, you should be worldly-wise to hit http://localhost:15672.</p> <p>NOTE: Consult someone who knows what they are doing if SSL is important to you. Borrowed some of this from <a href="http://weblogs.asp.net/jeffreyabecker/Using-SSL-client-certificates-for-authentication-with-RabbitMQ">here</a>.</p> <h4 id="troubleshooting">Troubleshooting</h4> <p>This might not work the first time.</p> <ul> <li>You might need to start SSL in the Erlang environment.Unshutwerl.exe, run <em>ssl:start().</em> That period is part of the syntax.</li> <li>Your rabbitmq.config file might point to the wrong path for your certs, considering you mistyped it like me : )</li> <li>Assuming you set your wiring to C:\RabbitMQ, trammels C:\RabbitMQ\Log\rabbit@<em>your_hostname_here</em>.log, squint up the errors you see</li> <li>Google virtually for other worldwide RabbitMQ or Erlang troubleshooting steps.</li> </ul> <p>Thankfully, Jason Morgan might be working on a DSC resource for this!</p> <p><img src="/images/rabbitmq/dscresource.png" alt="A DSC Resource for RabbitMQ?" /></p> <p>We have a RabbitMQ server up and running! What can we do with it?</p> <h3 id="managing-rabbitmq-with-rabbitmqtools">Managing RabbitMQ with RabbitMQTools</h3> <p>A while back, Mariusz Wojcik wrote a fairly well-constructed module for managing RabbitMQ via the REST API: RabbitMQTools. This fit the bill, but I needed HTTPS support, and I prefer a Credential parameter to a plaintext password.</p> <p>Download the <a href="https://github.com/RamblingCookieMonster/RabbitMQTools/archive/master.zip">fork of RabbitMQTools</a>, unblock the archive, and add it to one of your module paths.</p> <script src="https://gist.github.com/0771c69f3d408c44c754.js"> </script> <p>This lawmaking created a simple fanout exchange:</p> <p><a href="https://www.rabbitmq.com/tutorials/amqp-concepts.html"><img src="/images/rabbitmq/exchange-fanout.png" alt="Fanout exchange" /></a></p> <p>This ways any message we send to the mart is repeated to all queues unseat to that exchange. A broadcast, if you will.</p> <h3 id="sending-and-receiving-messages-with-psrabbitmq">Sending and receiving messages with PSRabbitMQ</h3> <p>RabbitMQTools is pretty handy, and you can plane send and receive messages with it, but it’s not quite as efficient as using the RabbitMQ .NET client. Chris Duck was generous unbearable to share some sanitized lawmaking for a RabbitMQ module, which I proceeded to rudely muck up : )</p> <p>Download <a href="https://github.com/RamblingCookieMonster/PSRabbitMQ">PSRabbitMQ</a>, unblock the <a href="https://github.com/RamblingCookieMonster/PSRabbitMq/archive/master.zip">archive</a>, and add it to one of your module paths.</p> <script src="https://gist.github.com/505b8440cf0a9e44ec5c.js"> </script> <h3 id="wrap">Wrap</h3> <p>A big thanks to Mariusz and Chris for RabbitMQTools and the RabbitMQ module, respectively.</p> <p>Definitely trammels out RabbitMQ if a messaging tool would goody your PowerShell and other solutions. At the very least, if you’re trying to write-up when orchestration pushers from VMware, Microsoft, BMC, Citrix, and the various other vendors, a little wits with RabbitMQ and <a href="https://ramblingcookiemonster.wordpress.com/2014/03/12/sql-for-powershell-for-sql-newbies/">SQL server</a> should alimony them at bay.</p> <p>That’s well-nigh it! You can get pretty deep lanugo the rabbit slum with messaging and RabbitMQ, the examples here just glanced the surface.</p> <h4 id="rambling-aside">Rambling aside</h4> <p>Congrats to June Blender et al for their recent MVP awards! June <a href="https://connect.microsoft.com/PowerShell/feedbackdetail/view/1351032/add-powershell-tab-and-examples-to-net-reference-pages-in-msdn">posted a connect issue</a> when in May to highlight the need for PowerShell examples in the MSDN .NET references, which would be veritably fantastic.</p> <p>On top of PowerShell examples in the .NET references, would love to see a tool that could generate usable (even if not optimal) PowerShell lawmaking from C# snippets. With so many third party solutions illustrating their .NET usage through C#, wouldn’t it be unconfined to have a tool to convert the <a href="http://www.rabbitmq.com/tutorials/tutorial-one-dotnet.html">RabbitMQ</a> and myriad other .NET examples to PowerShell? Hopefully we see something interesting from <a href="https://github.com/adamdriscoll/PowerShellCodeDomProvider">Adam Driscoll</a>, Microsoft, or the wider community.</p> <p>Cheers!</p> <p><a href="http://ramblingcookiemonster.github.io/RabbitMQ-Intro/">RabbitMQ and PowerShell</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on July 06, 2015.</p> http://ramblingcookiemonster.github.io/Decorating-Objects 2015-06-22T18:00:00+00:00 2015-06-22T18:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#type-names" id="markdown-toc-type-names">Type Names</a></li> <li><a href="#formatting-your-objects" id="markdown-toc-formatting-your-objects">Formatting Your Objects</a> <ul> <li><a href="#defaultdisplaypropertyset" id="markdown-toc-defaultdisplaypropertyset">DefaultDisplayPropertySet</a></li> <li><a href="#formatps1xml" id="markdown-toc-formatps1xml">Format.ps1xml</a></li> </ul> </li> <li><a href="#reusable-tools" id="markdown-toc-reusable-tools">Reusable Tools</a> <ul> <li><a href="#add-objectdetail" id="markdown-toc-add-objectdetail">Add-ObjectDetail</a></li> </ul> </li> </ul> </div> </section> <!-- /#table-of-contents --> <h3 id="rambling">Rambling</h3> <p>I recently ran a quick survey on source tenancy for IT professionals. While drafting up some notes on the <a href="http://ramblingcookiemonster.github.io/Source-Control-Survey/">results</a>, I was pleasantly surprised to find that Atlassian had a self-ruling <a href="https://www.atlassian.com/software/views/community-license-request">community license</a> for non-profits and charitable organizations.</p> <p>After an superstitious co-worker helped out with the implementation, <a href="https://www.atlassian.com/software/stash/">Stash</a>, a git-based source tenancy solution with a RESTful API, was up and running. Despite <a href="https://ramblingcookiemonster.wordpress.com/2015/02/07/rest-powershell-and-infoblox/">my thoughts on REST APIs</a>, I like to poke virtually unfamiliar technologies and solutions through PowerShell - time for a <a href="https://github.com/RamblingCookieMonster/PSStash">POC Stash PowerShell module</a>!</p> <p>After some quick and dirty code, I had output… but it was quite ugly:</p> <p><a href="/images/decorating/RepoUgly.png"><img src="/images/decorating/RepoUgly.png" alt="Ugly Repo Output" /></a></p> <p>Let’s squint at what we can do to tame this output into something increasingly readable, like this:</p> <p><a href="/images/decorating/Get-StashRepo.png"><img src="/images/decorating/Get-StashRepo.png" alt="Default Repo Properties" /></a></p> <p>Along the way, we’ll imbricate setting default properties, subtracting your own type names, and a tool to simplify all of this.</p> <h3 id="type-names">Type Names</h3> <p>One of the most important commands you learn in PowerShell is Get-Member. This can requite you a wealth of information on the objects you are working with, including the type of these objects:</p> <p><img src="/images/decorating/GetMember.png" alt="Get-Member" /></p> <p>In this case, we see that Get-Date gives us a System.DateTime.</p> <p>So, why would we want to add a custom type name to our objects? Jason Morgan pointed out two good reasons <a href="https://jasonspowershellblog.wordpress.com/2014/04/04/giving-type-names-to-your-custom-objects/">here</a>:</p> <ul> <li>Custom Formating: You can write (or borrow) ‘format’ files to pinpoint the default exhibit for your custom types.</li> <li>Object filtering and pipeline support: If your objects have a specific type, you can filter them thus and largest support pipeline input.</li> </ul> <p>The lawmaking isn’t pretty, but inserting a type name only takes a single line. Here’s a quick example:</p> <script src="https://gist.github.com/f83268daca1ed373f35a.js"> </script> <p><img src="/images/decorating/NonsenseType.png" alt="Nonsense type" /></p> <p>We have a type name in there, let’s take a squint at what we can do with it.</p> <h3 id="formatting-your-objects">Formatting Your Objects</h3> <p>If you use PowerShell, you know that objects don’t unchangingly show their hand. You might need to explore a bit with Get-Member, Select-Object, or <a href="http://www.powershellcookbook.com/recipe/bpqU/program-interactively-view-and-explore-objects">Show-Object</a>.</p> <p>When we run Get-ChildItem versus C:, we only see a few properties:</p> <p><img src="/images/decorating/DefaultGCIDisplay.png" alt="Nonsense type" /></p> <p>We know there are many increasingly - How does PowerShell know how to format these? Jeffrey Snover wrote <a href="http://blogs.msdn.com/b/powershell/archive/2010/02/18/psstandardmembers-the-stealth-property.aspx">a quick bit</a> on this a while back: PowerShell checks for format files, a DefaultDisplayPropertySet, and then falls when to all properties.</p> <p>Let’s squint at the first two.</p> <h4 id="defaultdisplaypropertyset">DefaultDisplayPropertySet</h4> <p>Like subtracting a type name, this is a bit ugly, but it’s pretty straightforward to copy, paste, and tweak as needed.</p> <p>Here’s a snippet well-timed from <a href="http://blogs.microsoft.co.il/scriptfanatic/2012/04/13/custom-objects-default-display-in-powershell-30/">Shay Levy’s abstraction</a>.</p> <script src="https://gist.github.com/f03aeeae705cbb218a83.js"> </script> <p>Whew! That was ugly, but it worked:</p> <p><img src="/images/decorating/DefaultDisplayPropertySet.png" alt="DefaultDisplayPropertySet" /></p> <p>Not only was this ugly, it was inefficient. We widow this to a single object, and would need to repeat the Add-Member line for every object we want to format. How can we get virtually this?</p> <h4 id="formatps1xml">Format.ps1xml</h4> <p>Time for some fun with XML! Run Get-Help <a href="https://technet.microsoft.com/en-us/library/hh847831.aspx">about_Format.ps1xml</a> for increasingly detailed help on this.</p> <p>Long story short, we can infringe a format.ps1xml file, modify it to meet our needs, and use it to pinpoint the default properties (and more) for any object of a unrepealable type. This was a big reason overdue why we wanted to add custom type names.</p> <p>You can swoop into <a href="https://msdn.microsoft.com/en-us/library/gg580944.aspx">MSDN</a>, or you can get started quickly by borrowing someone else’s work. This snippet will unshut up the format ps1xml for events:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">ise</span> <span class="nv">$PSHOME</span>\Event.Format.ps1xml </code></pre></div></div> <p>I cut out the shit I don’t superintendency well-nigh - everything but one view node. Let’s substitute out the TypeName for our own:</p> <div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;TypeName&gt;</span>Some.Nonsense.Typename<span class="nt">&lt;/TypeName&gt;</span> </code></pre></div></div> <p>We don’t want any grouping, so we can cut the GroupBy section out.</p> <p>Now we can just play with the table headers and table rows. <a href="https://gist.github.com/RamblingCookieMonster/10aeb2d4c41698cc2c86">Here’s the resulting ps1xml</a> - it might squint scary, but it’s literally reprinting and paste, with a few minor tweaks.</p> <p>We have a few options to get this format into our session. First, let’s just add the TypeData on the fly:</p> <script src="https://gist.github.com/dc7732c3e958828097e7.js"> </script> <p><img src="/images/decorating/NonsenseDisplay.png" alt="Nonsense Display" /></p> <p>It worked! We didn’t need to mess virtually with DefaultDisplayPropertySet for every single object, all we did was add a TypeName, and PowerShell took superintendency of the rest.</p> <p>If you’re writing a module, it’s plane simpler. Just specify the path to the format file(s) in the module manifest. Here’s an example from the <a href="https://github.com/RamblingCookieMonster/PSStash/blob/master/PSStash/PSStash.psd1">PSStash psd1 file</a>:</p> <div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Format files (.ps1xml) to be loaded when importing this module</span> FormatsToProcess <span class="o">=</span> @<span class="o">(</span><span class="s1">'PSStash.Format.ps1xml'</span><span class="o">)</span> </code></pre></div></div> <h3 id="reusable-tools">Reusable Tools</h3> <p>One of the unconfined benefits to lawmaking in unstipulated is the worthiness to utopian out seemingly ramified tasks into <a href="http://ramblingcookiemonster.github.io/Invoke-Ping/">reusable tools</a>. PowerShell was explicitly designed as a task-based language; for example, rather than needing to know the lawmaking overdue how to sort objects, we have the task-based Sort-Object writ in our toolbelt. The lawmaking to decorate objects whilom wasn’t pretty, let’s squint at a simple to use tool that memorandums out the wearisome bits.</p> <h4 id="add-objectdetail">Add-ObjectDetail</h4> <p>We can swash all the logic and .NET calls lanugo to a single function: <a href="https://raw.githubusercontent.com/RamblingCookieMonster/PSStash/master/PSStash/Private/Add-ObjectDetail.ps1">Add-ObjectDetail</a>. Download the ps1 and dot source it as needed. Here are a few examples:</p> <script src="https://gist.github.com/5382417d8d248a24b564.js"> </script> <p>We can add type names, set default properties, and add new properties. The last piece is increasingly for convenience; rather than calling Add-ObjectDetail withal with flipside function to add properties, we can do it all at once.</p> <p>That’s well-nigh it! You can use this to make your tools easier to use for end users and yourself. The function could likely use some work; it was written for the POC Stash module, but it seems to do the trick.</p> <p>Cheers!</p> <p><em>Disclaimer</em>: Thumbnail credit to <a href="http://www.pbs.org/parents/kitchenexplorers/2013/06/27/how-to-decorate-cookie-monster-cupcakes/">PBS</a></p> <p><a href="http://ramblingcookiemonster.github.io/Decorating-Objects/">Decorating PowerShell Objects</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on June 22, 2015.</p> http://ramblingcookiemonster.github.io/Trust-but-Verify 2015-06-08T22:00:00+00:00 2015-06-08T22:00:00+00:00 Warren F http://ramblingcookiemonster.github.io <section id="table-of-contents" class="toc"> <header> <h3><i class="fa fa-book"></i> Overview</h3> </header> <div id="drawer"> <ul id="markdown-toc"> <li><a href="#rambling" id="markdown-toc-rambling">Rambling</a></li> <li><a href="#what-could-go-wrong" id="markdown-toc-what-could-go-wrong">What Could Go Wrong?</a></li> <li><a href="#bad-input" id="markdown-toc-bad-input">Bad Input</a></li> <li><a href="#validation" id="markdown-toc-validation">Validation</a></li> <li><a href="#what-could-go-wrong-with-my-code" id="markdown-toc-what-could-go-wrong-with-my-code">What Could Go Wrong With My Code?</a></li> <li><a href="#something-went-wrong" id="markdown-toc-something-went-wrong">Something Went Wrong</a></li> <li><a href="#trust-but-verify" id="markdown-toc-trust-but-verify">Trust, but Verify</a></li> </ul> </div> </section> <!-- /#table-of-contents --> <h3 id="rambling">Rambling</h3> <p>A short while back, Adam Bertram was looking for suggestions on a theme for a PowerShell blog week. I suggested validation and error handling - these may seem dry, and are often overlooked, despite stuff veritably hair-trigger for reliability and consistency. June corrected my phrasing:</p> <p><img src="/images/error-handling/june.png" alt="Not dry" /></p> <p>June is right. The problem is, we often don’t realize how important these tasks are. In real life, we tend to take these for granted as well. We squint both ways surpassing crossing the road. We trammels our wallet surpassing heading to a cash-only spot. A doughboy taste-tests his craft. We don’t stop looking at the traffic light just considering it was untried 15 seconds ago. Many of these cases wilt second nature, and you may not realize you’re doing it.</p> <p>Yet often enough, someone writing lawmaking might forget to double trammels they have the expected results surpassing proceeding, might let their lawmaking run without an error occurs that could lead to disastrous results, or might use un-sanitized user input directly in a SQL query. All of these are solved problems, yet we might overlook them as ‘boring’ or ‘dry’.</p> <p>Let’s take a squint at how we can bring these issues to light surpassing they rationalization major problems.</p> <h3 id="what-could-go-wrong">What Could Go Wrong?</h3> <p>This is a good question to ask yourself as you write code, PowerShell or otherwise. What could intentionally or unintentionally go wrong? Ask this question often, and you will often end up with a increasingly reliable, consistent, and secure solution.</p> <p>Some variations to consider:</p> <ul> <li>What could go wrong… with the provided input?</li> <li>What could go wrong… with the data stuff processed?</li> <li>What could go wrong… with my expectations?</li> <li>What could go wrong… with the operating environment?</li> <li>What could go wrong… with my code?</li> </ul> <p>When you first start considering these questions, you might wonder if they are worth the uneaten time and thought. “What are the chances!” you might ask. If you find yourself taking shortcuts, be sure to consider the implications when something invariably goes wrong.</p> <p>I like to hang out in the <a href="http://slack.poshcode.org/">PowerShell Slack</a> / IRC community; you can ask questions, help others, and pick up ideas to take to your environment. I have seen panicked questions that turned out to be resume-generating-events. Don’t be wrung to make mistakes, it happens to everyone, and in a non-toxic environment these are often seen as learning opportunities, but be prudent. Consider the potential impact your lawmaking could have.</p> <p>A few examples:</p> <ul> <li>You’re only reading data… but that data feeds management decisions or processes lanugo the line. You might provide bad data to downstream processes.</li> <li>You’re waffly data… what if you transpiration the wrong data? Will you know it changed? How many merchantry processes or services will it impact?</li> <li>You’re removing data… you just deleted production data! Hopefully you have backups. Will you notice in time to restore without a significant impact?</li> <li>You exposed your service to injection… you suffer unintentional or intentional injection. Could be SQL injection, lawmaking injection, you name it. Not good!</li> </ul> <p>Let’s swoop into some worldwide scenarios. We won’t be worldly-wise to imbricate every imaginable scenario, so be sure to unchangingly ask <em>what could go wrong?</em></p> <h3 id="bad-input">Bad Input</h3> <p>The idea with input validation is that you should tenancy and limit the input that you take, to stave surprising outcomes. There are a number of options:</p> <ul> <li>Use built in PowerShell functionality for validation. Boe Prox wrote <a href="http://learn-powershell.net/2014/02/04/using-powershell-parameter-validation-to-make-your-day-easier/">a unconfined post</a> on this, and <a href="https://technet.microsoft.com/en-us/library/hh847743.aspx">about_Functions_Advanced_Parameters</a> has a few tips as well.</li> <li><a href="http://ramblingcookiemonster.github.io/Types-And-Enums/">Use an Enum</a> that restricts you to a few specific choices, or a strong type, such as <code class="highlighter-rouge">[int]</code> or <code class="highlighter-rouge">[string[]]</code> to indicate exactly what type of data to take in.</li> <li>Use other built in tools. If you are using this input in a SQL query lanugo the line, use safeguards like <a href="http://blog.codinghorror.com/give-me-parameterized-sql-or-give-me-death/">parameterized queries</a>, perhaps through <a href="https://github.com/RamblingCookieMonster/PowerShell/blob/master/Invoke-Sqlcmd2.ps1">Invoke-Sqlcmd2</a>.</li> <li>In some cases, you might need to indulge wrong-headed input, performing validation later in your code.</li> </ul> <p>These revolved virtually input validation, let’s swoop into validation in general.</p> <h3 id="validation">Validation</h3> <p>Validation can be used wideness wide swathes of your code. There are too many examples to count, wideness every imaginable scenario. Let’s list a few.</p> <ul> <li>Does the count of objects you get when meet your expectations? Did you get increasingly than expected? Did nothing come back? Did you plan to feed one Active Directory worth in for some changes but unwittingly get every single worth in the domain back?</li> <li>Your SQL server deployment system is ready to install SQL; is the SQL service worth it created older ready to use and replicated wideness Active Directory?</li> <li>Your vendor shipped shoddy code. You’re <a href="https://gallery.technet.microsoft.com/scriptcenter/Get-EvilProcess-Find-a8601566">hunting for runaway processes</a>. Do you verify the executable name, session idle time, and other bits, or do you skiver processes willy-nilly?</li> <li>You’re relying on a file stuff available. Do you seem that it’s there and that it’s ready to use immediately, or do you test and <a href="https://gallery.technet.microsoft.com/scriptcenter/Wait-Path-Wait-for-a-path-1393ef86">wait</a> for it to exist?</li> <li>You’re supposed to get a string back. Or a date. Or some other specific type. Do you verify you’re getting the right type surpassing working with it?</li> <li>You’re making changes to a set of systems. Would it help to <a href="http://ramblingcookiemonster.github.io/Invoke-Ping/">check connectivity</a> or make sure the right services are running surpassing starting?</li> </ul> <p>This all depends on your task at hand. You’re never going to find a list of everything to watch out for, but you can unchangingly ask yourself <em>what could go wrong?</em> Occasionally we have to ask this of our own expectations on how our lawmaking will behave.</p> <h3 id="what-could-go-wrong-with-my-code">What Could Go Wrong With My Code?</h3> <p>This is a tough one, and highlights the importance of testing your code. Here are a few mistakes I’ve seen. All of these swash lanugo to your lawmaking not doing what you think it’s doing.</p> <ul> <li>You’re writing a loop in lawmaking that will run in PowerShell 2, and you don’t test the loop variable first. In PowerShell 2 and earlier, a loop will run one time, <a href="http://stackoverflow.com/questions/21755825/why-is-it-possible-to-loop-through-a-null-array">even if you feed it $null</a>.</li> <li>You’re working in a PowerShell 2 environment again. This time, you’re taking whoopee based on the count of items you get back. Hopefully you didn’t depend on the count property; <a href="http://powershell.com/cs/blogs/tips/archive/2008/11/18/converting-results-into-arrays.aspx">in PowerShell 2</a>, if only a single object comes back, it very likely will not have a count property on it. This ways if you get one item back, and your logic tests <code class="highlighter-rouge">$item.count -eq 1</code>, or something withal those lines, you are out of luck.</li> <li>You’re handling errors with Try/Catch - awesome! Unfortunately, you forgot to tell PowerShell to gravity a ‘terminating’ error by specifying ErrorAction Stop; this ways you might never hit the reservation block. Be sure to read up on <a href="https://www.penflip.com/powershellorg/the-big-book-of-powershell-error-handling">error handling</a>.</li> <li>You’re handling errors with Try/Catch, and you made it to the reservation block! You reference <code class="highlighter-rouge">$Error[0]</code>. Turns out by some fluke flipside error squeaked its way in, pushing the error you superintendency well-nigh to <code class="highlighter-rouge">$Error[1]</code>.Unchanginglyrefer to the current error in the reservation woodcut as <code class="highlighter-rouge">$_</code>. Do not rely on <code class="highlighter-rouge">$Error</code>.</li> </ul> <p>There are a number of handy references on related topics:</p> <ul> <li>Michael Sorens’ <a href="https://www.simple-talk.com/sysadmin/powershell/a-plethora-of-powershell-pitfalls-part-2/">Plethora of PowerShell Pitfalls</a> series.</li> <li>PowerShell.org’s <a href="https://www.penflip.com/powershellorg/the-big-book-of-powershell-gotchas">BigTypesettingof PowerShell Gotchas</a></li> </ul> <p>So what do you do when you successfully snift something wrong?</p> <h3 id="something-went-wrong">Something Went Wrong</h3> <p>There are a number of ways to handle these scenarios:</p> <ul> <li>Error handling. <a href="https://www.penflip.com/powershellorg/the-big-book-of-powershell-error-handling">References</a> <a href="http://learn-powershell.net/2015/04/04/a-look-at-trycatch-in-powershell/">abound</a>. Long story short, get very familiar with <a href="https://technet.microsoft.com/en-us/library/hh847793.aspx">Try/Catch and Try/Catch/Finally</a>. Consider whether you should stop the unshortened function/script, ‘continue’ to the next item in a loop, or simply log the error and alimony on going.</li> <li>Logging. There are <a href="https://gallery.technet.microsoft.com/scriptcenter/PSLog-Send-messages-to-a-db389927">many</a> ways to <a href="https://gallery.technet.microsoft.com/scriptcenter/Enhanced-Script-Logging-27615f85">skin a cat</a>, including writing your own logging module. Logging unexpected scenarios and details on the operating environment when they occur can come in quite handy.</li> <li>Notification. If the process is important enough, or needs firsthand attention, consider some sort of notification. I tend to send <a href="https://gallery.technet.microsoft.com/scriptcenter/PowerShell-HTML-Notificatio-e1c5759d">HTML based e-mail</a>. You might have a monitoring system you could feed data into.</li> <li>Testing. This is a good way to squint for something going wrong with your code. Take a peak at <a href="http://ramblingcookiemonster.github.io/GitHub-Pester-AppVeyor/#pester">Pester</a>, it is quite valuable.</li> </ul> <h3 id="trust-but-verify">Trust, but Verify</h3> <p>One of the increasingly terrifying lines I hear nowadays is the old “oh, that will just be a couple lines of code!”</p> <p>You might trust your code. You might trust your environment. Regardless, you should unchangingly ask yourself <em>what could go wrong?</em> and verify your expectations. You might just save yourself, your employer, and your customers a major downtime, a massive data breach, or the many other potential outcomes that could occur when we make assumptions.</p> <p><a href="http://ramblingcookiemonster.github.io/Trust-but-Verify/">Trust&#44; but Verify</a> was originally published by Warren F at <a href="http://ramblingcookiemonster.github.io">Rambling Cookie Monster</a> on June 08, 2015.</p>